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Japanese Patent Office, which document is hereby incorporated 
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10 BACKGROUND OF THE INVENTION 

[0001] 

1. Field of the Invention 

The present invention relates to an image processing 
apparatus and method, and an imaging apparatus, in particular 
15 to an image processing apparatus and method, and an imaging 
apparatus which easily synthesize a plurality of image 
information data by using more various and complicated methods 
with a circuit size and manufacturing costs remaining reduced. 
[0002] 

20 2. Description of the Related Art 

Conventionally, a variety of methods have been proposed 
as an image processing method in which an image signal acquired 
by taking image via a video camera etc., is superimposed with 
other image data such as characters and graphics. 

25 [0003] 

For example, several methods have been proposed in which 
two OSD-ICs (On Screen Display-Integrated Circuit) connected 
in series respectively superimpose display data on inputted 
image signals, and a microcomputer controls operation of these 
30 two OSD-ICs so as to instantly switch the display data to be 
superimposed on the image signals, and a plurality of display 
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data are simultaneously displayed on the image signal (see 
Japanese Laid Open Patent No. H10-150609, pp. 3-6, and Fig. 
1, for example) . 
[0004] 

5 In addition, another method has been proposed in which 

an image signal to be superimposed and a superimposing image 
signal are respectively multiplied by a predetermined 
coefficient, then these two signals are added together, to 
thereby specify a synthesizing ratioof two signals arbitrarily . 

10 [0005] 

In these years, as a performance of an image processing 
apparatus and an image processing technique have been improved, 
there is increasingly a need for sophisticatedly superimposing 
image information data, such as various characters and graphics, 
15 on an image signal by adopting various methods. 
[0006] 

However, a problem in such a method as mentioned above 
is that the image signal can be superimposed only by a 
comparatively simple method. For example, in the case of a 
20 method of using two OSD -ICs connected in series, a synthesizing 
ratio of the image signal to be superimposed to the superimposing 
display data is always one-to-one and therefore the synthesizing 
ratio cannot be adjusted. 
[0007] 

25 When the method of multiplying the coefficient with each 

signal is used, a control unit for controlling superimposition 
of display data can adjust the synthesizing ratio of two signals . 
However, for example, even when it is desired to change a part 
of the display data, an OS D- 1 C must control to prepare and display 

30 a new superimposing display data for one screen. Thus, there 
is a problem in that a processing load in the OSD-IC increases 
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and its manufacturing costs may also increase. In addition, 
there is another problem with this method that since a rewrite 
timing of the superimposing display data is limited to, for 
example, a vertical blanking period, a horizontal blanking 
5 period, etc., it is necessary for the OSD-IC to carry out the 
process at a high speed, thus increasing the manufacturing costs 
further. 
[0008] 

Further, even in the case where the method of using a 
10 plurality of OSD-ICs connected in series is combined with the 
method of multiplying the coefficient, it is impossible to 
control the superimposing display data for each partial area 
of the screen. Therefore, there is another problem that , when 
switching the display data, a display data for one screen must 
15 be prepared to thereby increases the processing load and the 
circuit size. 
[0009] 

For example, when the display data are superimposed on 
two mutually different areas, such as a first area and a second 

20 area of an image data for one screen corresponding to the image 
signal, if the display data only in the first area is switched, 
it is necessary to prepare three OSD-ICs, which are a first 
OSD-IC for superimposing the display data displayed on the first 
area before the switching, a second OSD-IC for superimposing 

25 the display data displayed on the second area, and a third OSD-IC 
for superimposing the display data displayed on the first area 
after the switching. Therefore, in this method, in order to 
carry out a complicated method of switching the superimposing 
display data, it is necessary to prepare a lot of OSD-ICs 

30 beforehand so as correspond to the degree of complexity, so 
that not only the load of processing increases but also the 
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circuit size becomes larger, which leads to increased 
manufacturing costs. 

SUMMARY OF THE INVENTION 

5 [0010] 

In view of the above situation, the present invention 
has been invented to easily synthesize a plurality of image 
information data using various and complicated method, with 
the reduced circuit size and manufacturing costs. 
10 [0011] 

An image processing apparatus according to the present 
invention includes: synthesizing image information data 
holding means for holding a plurality of image information data 
to be synthesized; synthesis control means for controlling 

15 synthesis of the plurality of synthesizing image information 
data held in the synthesizing image information data holding 
means and input image information data, for every arbitrary 
area of an input image corresponding to the input image 
information data; and image information data synthesis means 

20 for synthesizing the input image information data and the 
synthesizing image information data according to the control 
by means of the synthesis control means. 
. [0012] 

The synthesizing image holding means can hold the 
25 synthesizing image information data as data on a pixel-by-pixel 
basis. The synthesizing image holding means can hold 
information data obtained by arranging the synthesizing image 
information data in a table. 
[0013] 

30 The synthesis control means includes control information 

data holding means for holding control information data about 
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control of synthesis of the synthesizing image information data 
and the input image information data, so as to control the 
synthesis of the synthesizing image information data and the 
input image information data according to the control 
5 information data held in the control information data holding 
means. 
[0014] 

The control information data is an information data for 
specifying, in an arbitrary area, synthesizing image 

10 information data corresponding to a synthesizing image to be 
superimposed on the input image by selecting it from the 
plurality of synthesizing image information data held in the 
synthesizing image information data holding means. Based on 
the control information data, the synthesis control means 

15 determines whether or not each of the plurality of synthesizing 
image information data held in the synthesizing image 
information data holding means is synthesized with the input 
image information data, such that the synthesizing image 
information data which is determined to be synthesized is 

20 controlled to be synthesized with the input image information 
data. Based on the control by means of the synthesis control 
means, the image information data synthesis means synthesizes 
the input image information data and the synthesizing image 
information data so as to superimpose the input image on the 

25 synthesizing image and vice versa in an arbitrary area. 
[0015] 

The control information data is an information data for 
specifying a synthesizing ratio of respective image information 
data in arbitrary areas when mixing the input image with the 
30 synthesizing image corresponding to the plurality of 
synthesizing image information data held in the synthesizing 
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image information data holding means. Based on the control 
information data, the synthesis control means controls the 
plurality of synthesizing image information data held at the 
synthesizing image information data holding means so as to be 
5 synthesized with the input image information data at the 
specified synthesizing ratio. Based on the control by means 
of the synthesis control means , the image information data 
synthesis means synthesizes the input image information data 
and the synthesizing image information data so as to mix the 
10 input image and the synthesizing image at the specified 
synthesizing ratio in the arbitrary areas. 
[0016] 

The synthesis control means further includes, in the 
arbitrary areas, graphics determination means for determining 

15 whether or not graphics exist in the synthesizing image 
corresponding to the synthesizing image information data 
synthesized with the input image information data. When it 
is determined that there are not graphics by way of the 
determination by means of the graphics determination means, 

20 it is possible to control the synthesizing image information 
data so as not to be synthesized with the input image information 
data . 
[0017] 

An amount of data of the control information data can 
25 be smaller than an amount of data of the synthesizing image 
information data held in the synthesizing image information 
data holding means. The control information data can be an 
information data on a pixel-by-pixel basis. The control 
information data can be an information data obtained by 
30 arranging, in a table, transition points where control changes . 
[0018] 



Further the image processing apparatus further includes 
address information data generation means for generating the 
address information data which indicates a location in a screen 
for the input image, and based on the address information data 
5 generated by the address information data generation means, 
the synthesis control means can control the synthesis of the 
input image information data and the plurality of synthesizing 
image information data held in the synthesizing image 
information data holding means. such that synthesis locations 
10 of the synthesizing image information data and input image 
information data may be positioned properly. 
[0019] 

Further the image processing apparatus includes 
synchronizing signal separation means for separating a 

15 synchronizing signal added to the input image information data, 
the address information data generation means can generate 
address information data, based on the synchronizing signal 
separated from input image information data by the synchronizing 
signal separation means. 

20 [0020] 

The image processing method of the present invention 
includes: an synthesizing image information data hold control 
step of controlling hold of a plurality of synthesizing image - 
information data; a synthesis control step of controlling 

25 synthesis of the input image information data and the plurality 
of synthesizing image information data which are controlled 
andheldby way of a process of the synthesizing image information 
data hold control step, for every arbitrary area of an input 
image corresponding to an input image information data; and 

30 an image information data synthesis step of synthesizing the . 
input image information data and the synthesizing image 
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information data, according to the control by way of the process 
of the synthesis control step. 
[0021] 

The synthesizing image hold control step can control and 
5 hold the synthesizing image information data as data on a 
pixel-by-pixel basis. The synthesizing image hold control 
step can control and hold information data obtained by arranging 
the synthesizing image information data in a table. 
[0022] 

10 The synthesis control step includes a control information 

data hold control step of controlling hold of the control 
information data about control of synthesis of the synthesizing 
image information data and the input image information data, 
so as to control the synthesis of the synthesizing image 

15 information data and input image information data according 
to the control information data which is controlled and held 
by way of the process of the control information data hold control 
step . 
[0023] 

20 In the arbitrary areas, according to the control 

information data for specifying the synthesizing image 
information data corresponding to the synthesizing image which 
is superimposed on the input image, by selecting it from the 
plurality of synthesizing image information data which are 

25 controlled and held by way of the process of the synthesizing 
image information data hold control step, the synthesis control 
step determines whether each of the plurality of synthesizing 
image information data which are controlled and held by way 
of the process of the synthesizing image information data hold 

30 control step is synthesized with the input image information 
data or not. The synthesizing image information data which 



is determined to be synthesized is controlled so as to be 
synthesized with the input image information data. Based on 
the control by way of the process of the synthesis control step, 
the image information data synthesis step synthesizes the input 
5 image information data and the synthesizing image information 
data so as to superimpose the input image on the synthesizing 
image and vice versa in the arbitrary areas. 
[0024] 

In the arbitrary areas, the synthesis control step 

10 controls the plurality of synthesizing image information data 
which are controlled and held by way of the process of the 
synthesizing image information data hold control step so as 
to be synthesized with the input image information data at the 
specified synthesizing ratio, according to the control 

15 information data for specifying the synthesizing ratio of each 
image information data when mixing, with the input image, the 
synthesizing image corresponding to the plurality of 
synthesizing image information data which are controlled and 
held by way of the process of the synthesizing image information 

20 data hold control step. Based on the control by way of the 
process of the synthesis control step, the image information 
data synthesis step synthesizes the input image information 
data and the synthesizing image information data so as to mix 
the input image and the synthesizing image at the specified 

25 synthesizing ratio in the arbitrary areas. 
[0025] 

The process of the synthesis control step further includes 
a graphics determination step of determining whether or not 
graphics exist in the synthesizing image corresponding to the 
30 synthesizing image information data which is synthesized with 
the input image information data in the arbitrary areas. When 



it is determined that there are not graphics through the 
determination by way of the process of the graphics 
determination step, it is possible to control the synthesizing 
image information data so as not to be synthesized with the 
5 input image information data. 
[0026] 

The amount of data of the control information data can 
be smaller than the amount of data of the synthesizing image 
information data which is controlled and held by way of the 

10 process of the synthesizing image information data hold control 
step. The control information data can be an information data 
on a pixel-by-pixel basis. The control information data can 
be an information data obtained by arranging, in a table, the 
transition points where the control changes. 

15 [0027] 

Further the method includes an address information data 
generation step of generating the address information data which 
indicates a location in a screen for the input image, and based 
on the address information data generated by way of the process 

20 of the address information data generation step, the synthesis 
control step can control the synthesis of the input image 
information data and the plurality of synthesizing image 
information data which are controlled and held by way of the 
process of the synthesizing image information data hold control 

25 step, so that the synthesis locations of the synthesizing image 
information data and the input image information data may be 
positioned properly. 
[0028] 

Further the method includes a synchronizing signal 
30 separation step of separating a synchronizing signal added to 
the input image information data, the address information data 
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generation step can generate address information data, based 
on the synchronizing signal separated from input image 
information data by way of the process of the synchronizing 
signal separation step. 
5 [0029] 

An imaging apparatus according to the present invention 
includes: imaging means for imaging a photographic subject and 
capturing an taken image information data which is an acquired 
image information data; an synthesizing image information data 

10 holding means for holding a plurality of synthesizing image 
information data which are synthesized with the taken image 
information data captured by the imaging means; synthesis 
control means for controlling synthesis of the taken image 
information data and the plurality of synthesizing image 

15 information data which are held in the synthesizing image 
information data holding means, for every arbitrary area of 
the taken image corresponding to the taken image information 
data; and image information data synthesis means for 
synthesizing the taken image information data and the 

20 synthesizing image information data according to control by 
means of the synthesis control means. 
[0030] 

According to the present invention, the plurality of 
synthesizing image information data are held; the synthesis 

25 of the plurality of synthesizing image information data and 
input image information data is controlled for every arbitrary 
area of the input image corresponding to input image information 
data; based on the control, the input image information data 
and the synthesizing image information data are synthesized. 

30 [0031] 

As described above, according to the present invention, 
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the plurality of image information data can be synthesized. 
In particular, the plurality of image information data can be 
easily synthesized by using various and complicated methods, 
with the circuit size and the manufacturing costs remaining 
5 reduced. 

BRIEF DESCRIPTION OF THE DRAWINGS 
Fig. 1 is a block diagram showing an example of structure 
of an image synthesizing apparatus to which the present 
10 invention is applied; 

Figs. 2A and 2B are charts showing an example of 
arrangement of a storage area of a first plane memory of Fig. 
1; 

Figs. 3A and 3B are charts showing an example of 
15 arrangement of a storage area of a superposing order control 
unit of Fig. 1; 

Fig. 4 is a flow chart for explaining an image synthesis 
process by means of the image synthesizing apparatus of Fig. 
1; 

20 Fig. 5 is another flow chart following the flow chart 

in Fig. 4, for explaining the image synthesis process by means 
the image synthesizing apparatus of Fig. 1; 

Figs . 6A to 6C are views showing an example of image which 
is synthesized by the image synthesizing apparatus of Fig. 1; 
25 Figs. 7A to 7C are views showing an example of the way 

in which an image is synthesized by the image synthesizing 
apparatus of Fig. 1; 

Fig. 8 is a block diagram showing another example of 
structure of the image synthesizing apparatus to which the 
30 present invention is applied; 

Figs . 9A and 9B are views showing an example of arrangement 

12 



of a storage area of a synthesizing ratio control unit in Fig. 
8; 

Fig. 10 is a flow chart for explaining an image synthesis 
process by means the image synthesizing apparatus of Fig. 8; 
5 Fig. 11 is a flow chart following that of Fig. 10 for 

explaining the image synthesis process by means of the image 
synthesizing apparatus of Fig. 8; 

Figs. 12A to 12C are views showing an example of image 
which is synthesized by the image synthesizing apparatus of 
10 Fig. 8; 

Figs. 13A to 13C are views showing an example of the way 
in which the image is synthesized by means of the image 
synthesizing apparatus of Fig. 8; 

Figs. 14A and 14B are views showing an example of the 
15 way in which the image is synthesized by means of the image 
synthesizing apparatus of Fig. 8; 

Fig. 15 is a block diagram showing an example of structure 
of a camcorder to which the present invention is applied; and 

Fig. 16 is a block diagram showing an example of structure 
20 of a personal computer. 

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS 
[0032] 

Hereafter, the preferred embodiments of the present 
25 invention will be described, and the followings are 
relationships between a constituent element recited in a claim 
and an example of the preferred embodiments in accordance with 
the present invention. The description is for confirming that 
an example which supports the invention recited in the claim 
30 is recited in the description of the preferred embodiments of 
the present invention. Therefore, if there is an example which 
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is recited in the preferred embodiments of the invention and 
not recited herein as one corresponding to constituent element , 
it does not mean that the example does not correspond to the 
constituent element. In contrast, if the example is recited 
5 as one corresponding to the constituent element, it does not 
mean that the example does not correspond to any constituent 
element other than the constituent elements. 
[0033] 

Further, this description does not mean that the invention 
10 corresponding to the example recited in the preferred 
embodiments of the invention is recited in all claims . In other 
words, this description corresponds to the example recited in 
the description of the preferred embodiments and does not deny 
existence of the invention which is not recited in a claim of 
15 this application, i.e. existence of the invention to be divided 
into a divisional application or to be added through amendments 
in the future. 
[0034] 

In a first aspect of the present invention, an image 
20 processing apparatus (for example, an image synthesizing 
apparatus 1 of Fig. 1 or an image synthesizing apparatus 150 
of Fig. 8) in which an input image information data which is 
an image information data contained in an input image signal 
is synthesized with an synthesizing image information data which 
25 is an image information data different from the input image 
information data, the image processing apparatus comprises: 
synthesizing image information data holding means for holding 
a plurality of the synthesizing image information data (for 
example, a first plane memory 18 and a second plane memory 19 
30 of Fig. 1 or 8); synthesis control means (for example, a 
superposing order control unit 20 of Fig. 1 or a synthesizing 



ratio control unit 152 of Fig. 8) for controlling synthesis 
of the input image information data and the plurality of the 
input synthesizing image information data which are held in 
the synthesizing image information data holdingmeans, for every 
5 arbitrary area (for example, a area 142 through a area 144 of 
Fig. 7C, or areas 282 through 284 of Fig. 14B) of the input 
image (for example, an input image 91 of Fig. 6A or an input 
image 211 of Fig. 12A) corresponding to the input image 
information data; and image information data synthesis means 

10 for synthesizing the input image information data and the 
synthesizing image information data (for example, switch 
circuits 21 and 22 of Fig. 1, or multipliers 153 through 155, 
and an adder 15 6 of Fig. 8) according to the control by means 
of the synthesis control means. 

15 [0035] 

In a second aspect of the invention, the image processing 
apparatus as recited in the first aspect, the synthesizing image 
holding means holds the synthesizing image information data 
as a data on a pixel-by-pixel basis (for example, a data 31 
20 on a pixel-by-pixel basis in Fig. 2A) . 
[0036] 

In a third aspect of the image processing apparatus as 
recited in the first aspect, the synthesizing image holding 
means holds information data obtained by arranging the 
25 synthesizing image information data in a table (for example, 
a table 40 of Fig. 2B) . 
[0037] 

In a forth aspect of the image processing apparatus as 
recited in the first aspect, the synthesis control means 
30 includes control information data holding means (for example, 
a memory unit 20A of Fig. 1 or a memory unit 152A of Fig. 8) 
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for holding control information data about control of synthesis 
of the synthesizing image information data and the input image 
information data, so as to control the synthesis of the 
synthesizing image information data and the input image 
5 information data according to the control information data held 
in the control information data holding means. 
[0038] 

In a fifth aspect of the image processing apparatus as 
recited in the forth aspect, the control information data is 

10 information data (for example, superposition control 
information data corresponding to an image 121 of Fig. 7A) for 
specifying, in the arbitrary area, the synthesizing image 
information data corresponding to a synthesizing image to be 
superimposed on the input image by selecting it from the 

15 plurality of synthesizing image information data held in the 
synthesizing image information data holding means, based on 
the control information data, the synthesis control means 
determines whether or not each of the plurality of synthesizing 
image information data held in the synthesizing image 

20 information data holding means is synthesized with the input 
image information data (for example, step S13 and step S15 of 
Fig. 5), such that the synthesizing image information data which 
has been determined to be synthesized is controlled to be 
synthesized with the input image information data, and based 

25 on the control by means of the synthesis control means, the 
image information data synthesis means synthesizes the input 
image information data and the synthesizing image information 
data so as to superimpose the input image on the synthesizing 
image and vice versa in the arbitrary area (for example, step 

30 S14 and step S16 of Fig. 5) . 
[0039] 



In a sixth aspect of the image processing apparatus as 
recited in the forth aspect, the control information data is 
information data (for example, synthesizing ratio control 
information data corresponding to an image 241 of Fig, 13A) 
5 for specifying the synthesizing ratio of each image information 
data in the arbitrary area, when mixing the input image with 
the synthesizing image corresponding to the plurality of 
synthesizing image information data held in the synthesizing 
image information data holding means, based on the control 

10 information data, the synthesis control means controls the 
plurality of synthesizing image information data held at the 
synthesizing image information data holding means so as to be 
synthesized with the input image information data at the 
specified synthesizing ratio (for example, step S53 of Fig. 

15 11) , and based on the control by means of the synthesis control 
means, the image information data synthesis means synthesizes 
the input image information data and the synthesizing image 
information data so as tomix the input image and the synthesizing 
image at the specified synthesizing ratio in the arbitrary area 

20 (for example, step S55 through step S57 of Fig. 11) . 
[0040] 

In a seventh aspect of the image processing apparatus 
as recited in the forth aspect, the synthesis control means 
further includes, in the arbitrary area, graphics determination 

25 means (for example, a synthesizing ratio control unit 152 of 
Fig. 8 for performing a process in step S54 of Fig. 11) for 
determining whether or not graphics exist in the synthesizing 
image corresponding to the synthesizing image information data 
synthesized with the input image information data, and when 

30 it is determined that there are not graphics by way of the 
determination through the graphics determination means, the 
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synthesis control means controls the synthesizing image 
information data so as not to be synthesized with the input 
image information data (Step S54 and step S56 of Fig. 11) . 
[0041] 

5 In an eighth aspect of the image processing apparatus 

as recited in the forth aspect, an amount of data of the control 
information data is smaller (for example, Fig. 3A) than an amount 
of data of the synthesizing image information data held in the 
synthesizing image information data holding means. 
10 [0042] 

In an ninth aspect of the image processing apparatus as 
recited in the forth aspect, the control information data is 
an information data on a pixel-by-pixel basis (for example, 
superposition control information data 51 on a pixel-by-pixel 
15 basis in Fig. 3A) . 
[0043] 

In a tenth aspect of the image processing apparatus as 
recited in the forth aspect, the control information data is 
an information data obtained by arranging transition points 
20 where control changes, in a table (for example, a table 7 0 in 
Fig. 3B) . 
[0044] 

In an eleventh aspect of the image processing apparatus 
as recited in the first aspect, the image processing apparatus 

25 further includes address information data generationmeans (for 
example, an address generation counter 12 of Fig. 1 or Fig. 
8) for generating the address information data which indicates 
a location in a screen for the input image, wherein based on 
the address information data generated by the address 

30 information data generation means, the synthesis control means 
controls the synthesis of the input image information data and 
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the plurality of synthesizing image information data held in 
the synthesizing image information data holding means such that 
synthesis locations of the synthesizing image information data 
and the input image information data may be positioned properly . 
5 [0045] 

In a twelfth aspect of the image processing apparatus 
as recited in the eleventh aspect, the image processing 
apparatus further includes synchronizing signal separation 
means (for example, a synchronizing signal separation 

10 processing unit 11 of Fig. 1 or Fig. 8) for separating a* 
synchronizing signal added to the input image information data, 
wherein the address information data generationmeans generates 
the address information data, based on the synchronizing signal 
separated from the input image information data by the 

15 synchronizing signal separation means. 
[0046] 

In a thirteenth aspect of the present invention, an image 
processing method for an image processing apparatus (for example, 
the image synthesizing apparatus 1 of Fig. 1 or the image 

20 synthesizing apparatus 150 of Fig. 8) in which an synthesizing 
image information data which is an image information data and 
different from an input image information data is synthesized 
with the input image information data . which is an image 
information data contained in an input image signal, the image 

25 processing method includes: synthesizing image information 
data hold control step of controlling hold of a plurality of 
the synthesizing image information data (for example, step S3 
and step S6 of Fig. 4, or step S43 and step S46 of Fig. 10); 
a synthesis control step of controlling synthesis of the input 

30 image information data and the plurality of synthesizing image 
information data which are controlled and held by way of a process 
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of the synthesizing image information data hold control step, 
for every arbitrary area (for example, the area 142 through 
area 144 of Fig. 7C, or areas 282 through 284 of Fig. 14B) of 
an input image (for example, the input image 91 of Fig. 6A or 
5 the input image 211 of Fig. 12A) corresponding to the input 
image information data (for example, step S13 and step S15 of 
Fig. 5, or step S53 of Fig. 11) ; and an image information data 
synthesis step of synthesizing the input image information data 
and the synthesizing image information data, according to the 
10 control by way of the process of the synthesis control step 

(for example, step S14 and step S16 of Fig . 5, or step S55 through 
step S57 of Fig. 11) . 

[0047] 

In a fourteenth aspect of the image processing method 
15 as recited in the thirteenth aspect, the synthesizing image 
hold control step controls and holds the synthesizing image 
information data as a data on a pixel -by-pixel basis (for example, 
the data 31 on a pixel-by-pixel basis of Fig. 2A) . 
[0048] 

20 In a fifteenth aspect of the image processing method as 

recited in the thirteenth aspect, the synthesizing image hold 
control step controls and holds information data obtained by 
arranging the synthesizing image information data in a table 
(for example, the table 40 of Fig. 2B) . 

25 [0049] 

In a sixteenth aspect of the image processing method as 
recited in the thirteenth aspect, the synthesis control step 
includes a control information data hold control step of 
controlling hold of the control information data about the 
30 control of the synthesis of the synthesizing image information 
data and the input image information data (for example, step 

20 



S9 of Fig. 4 or step S49 of Fig. 10) , and based on the control 
information data which is controlled and held by way of the 
process of the control information data hold control step, the 
synthesis control step controls the synthesis of the 
5 synthesizing image information data and the input image 
information data. 
[0050] 

In a seventeenth aspect of the image processing method 
as recited in the sixteenth aspect, the synthesis control step 

10 determines, in the arbitrary area, whether each of the plurality 
of synthesizing image information data which are controlled 
and held by way of the process of the synthesizing image 
information data hold control step is synthesized with the input 
image information data or not (for example, step S13 and step 

15 S15 of Fig. 5) , according to the control information data (for 
example, the superposition control information data 
corresponding to the image 121 of Fig. 7A) for specifying the 
synthesizing image information data corresponding to the 
synthesizing image which is superimposed on the input image, 

20 by selecting it from the plurality of synthesizing image 
information data which are controlled and held by way of the 
process of the synthesizing image information data hold control 
step, the synthesizing image information data which has been 
determined to be synthesized is controlled so as to be 

25 synthesized with the input image information data, and based 
on the control by way of the process of the synthesis control 
step, the image information data synthesis step synthesizes 
the input image information data and the synthesizing image 
information data so as to superimpose the input image on the 

30 synthesizing image and vice versa in the arbitrary area (for 
example, step S14 and step S16 of Fig. 5) . 
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[0051] 

In an eighteenth aspect of the image processing method 
as recited in the sixteenth aspect, the synthesis control step 
controls, in the arbitrary area, the plurality of synthesizing 
5 image information data which are controlled and held by way 
of the process of the synthesizing image information data hold 
control step so as to be synthesized with the input image 
information data at the specified synthesizing ratio (for 
example, step S53 of Fig. 11), according to the control 

10 information data (for example, the synthesizing ratio control 
information data corresponding to the image 241 of Fig. 13A) 
for specifying the synthesizing ratio of each image information 
data when mixing, with the input image, the synthesizing image 
corresponding to the plurality of synthesizing image 

15 information data which are controlled and held by way of the 
process of the synthesizing image information data hold control 
step, and based on the control by way of the process of the 
synthesis control step, the image information data synthesis 
step synthesizes the input image information data and the 

20 synthesizing image information data so as to mix the input image 
and the synthesizing image at the specified synthesizing ratio 
in the arbitrary area (for example, step S55 through step S57 
of Fig. 11) 
[0052] 

25 In an nineteenth aspect of the image processing method 

as recited in the sixteenth aspect, the process of the synthesis 
control step further includes a graphics determination step 
(for example, step S54 of Fig. 11) of determining whether or 
not graphics exist in the synthesizing image corresponding to 

30 the synthesizing image information data which is synthesized 
with the input image information data in the arbitrary area, 
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and when it is determined that there are not the graphics through 
the determination by way of the process of the graphics 
determination step, the process of the synthesis control step 
controls the synthesizing image information data so as not to 
5 be synthesized with the input image information data (Step S56 
of Fig. 11) . 
[0053] 

In a twentieth aspect of the image processing method as 
recited in the sixteenth aspect , the amount of data of the control 
10 information data is smaller than the amount of data of the 
synthesizing image information data which is controlled and 
held by way of the process of the synthesizing image information 
data hold control step (for example, Fig. 3A) . 
[0054] 

15 In a twenty-first aspect of the image processing method 

as recited in the sixteenth aspect, the control information 
data is an information data on a pixel-by-pixel basis (for 
example, the superposition control information data 51 on a 
pixel-by-pixel basis in Fig. 3A) . 

20 [0055] 

In a twenty-second aspect of the image processing method 
as recited in the sixteenth aspect, the control information 
data is an information data obtained by arranging transition 
points where'the control changes, in a table (for example, the 
25 table 70 of Fig. 3B) . ' 
[0056] 

In a twenty-third aspect of the image processing method 
as recited in the thirteenth aspect, the image processing method 
further includes an address information data generation step 
30 (for example, step S12 of Fig. 5 or step S52 of Fig. 11) of 
generating the address information data which indicates a 
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location in a screen for the input image, wherein based on the 
address information data generated by way of the process of 
the address information data generation step, the synthesis 
control step controls the synthesis of the input image 
information data and the plurality of synthesizing image 
information data which are controlled and held by way of the 
process of the synthesizing image information data hold control 
step, so that the synthesis locations of the synthesizing image 
information data and the input image information data may be 
positioned properly. 
[0057] 

In a twenty-forth aspect of the image processing method 
as recited in the twenty-third aspect, the image processing 
method further includes a synchronizing signal separation step 

(for example, step Sll of Fig. 5 or step S51 of Fig. 11) of 
separating a synchronizing signal added to the input image 
information data, wherein the address information data 
generation step generates the address information data, based 
on the synchronizing signal separated from the input image 
information data by way of the process of the synchronizing 
signal separation step. 

[0058] 

In a twenty-fifth aspect of the present invention, an 
imaging apparatus (for example, a camcorder 300 of Fig. 15) 
includes: imaging means (for example, a camera unit 310 of Fig. 
15) for imaging a photographic subject and capturing a taken 
image information data which is an acquired image information 
data; synthesizing image information data holding means (for 
example, the first plane memory 18 and the second plane memory 
19 of Fig. 1 or Fig. 8) for holding a plurality of synthesizing 
image information data which are synthesized with the taken 
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image information data captured by the imaging means; synthesis 
control means (for example, the superposing order control unit 
20 of Fig. 1 or the synthesizing ratio control unit 152 of Fig. 
8) for controlling synthesis of the taken image information 
5 data and the plurality of synthesizing image information data 
which are held in the synthesizing image information data 
holding means, for every arbitrary area of the taken image 
corresponding to the taken image information data; and image 
information data synthesis means (for example, the switch 

10 circuits 21 and 22 of Fig. 1, or the multipliers 153 through 
155 and the adder 156 of Fig. 8) for synthesizing the taken 
image information data and the synthesizing image information 
data according to control by means of the synthesis control 
means . 

15 [0059] 

Hereafter, the preferred embodiments according to the 
present invention will be described with reference to the 
accompanying drawings. Fig. 1 is a block diagram showing an 
example of structure of an image synthesizing apparatus to which 
20 the present invention is applied. 
[0060] 

In Fig. 1, an image synthesizing apparatus 1 is an 
apparatus in which, into a part or the whole of an image 
corresponding to an input image signal, such as an image signal 

25 included in a television signal corresponding to an NTSC 
(National Television Standards Committee) standard or the PAL 
(Phase Alternationby Line) standard, for example, another image 
showing a character, graphics, etc. is inserted, so that the 
resulting image is outputted as an output image signal. In 

30 other words, in the image synthesizing apparatus 1, the 
synthesizing image information data which are other image 
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information data are selectively synthesized with an input image 
information data contained in the input image signal, so as 
to be outputted. At this stage, the image synthesizing 
apparatus 1 can synthesize the image information data such that, 
5 in a plurality of areas in the screen of the image corresponding 
to the input image signal, the character, the graphics, etc. 
may be synthesized independently of one another for each area. 
For example, the image synthesizing apparatus 1 can change a 
part of another image inserted into the image corresponding 
10 to the input image signal, and further switch the image to still 
another image, or move it to another position. 
[0061] 

Although not shown, the synchronizing signal separation 
processing unit 11 of the image synthesizing apparatus 1 

15 includes an amplitude separationprocessingunit and a frequency 
separation processing unit, extracts a vertical synchronizing 
signal and a horizontal synchronizing signal from an input image 
signal, and supplies the synchronizing signals to an address 
generation counter 12 . For example, when the input image signal 

20 is an image signal corresponding to the NTSC standard, the input 
image signal includes a vertical synchronizing signal whose 
frequency is approximately 60 Hz and a horizontal synchronizing 
signal whose frequency is approximately 15.75 kHz. Firstly, 
the synchronizing signal separation processing unit 11 extracts 

25 a synchronizing signal containing both the vertical 
synchronizing signals and the horizontal synchronizing signal 
from the input image signal by means of the amplitude separation 
processing unit. Then, the synchronizing signal separation 
processing unit 11 separates the vertical synchronizing signal 

30 and the horizontal synchronizing signal from the extracted 
synchronizing signal by means of the frequency separation 
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processing unit, so that each of the vertical and horizontal 
synchronizing signals is outputted to the address generation 
counter 12. 
[0062] 

5 In addition, the synchronizing signal separation 

processing unit 11 may only extract a synchronizing signal 
contained in an input image signal, so that it may have any 
construction other than the construction as described above. 
[0063] 

10 The address generation counter 12 includes therein a 

counter circuit, counts the number of pulses of the acquired 
vertical synchronizing signal and horizontal synchronizing 
signal, generates the address information data based on these 
synchronizing signals, and provides the address information 

15 data to a control unit 13, the first plane memory 18, the second 
plane memory 19, and the superposing order control unit 20. 
[0064] 

The address information data is a position information 
data in an image for one screen of the input image information 

20 data contained in the input image signal, and also is an 
information data for making the synthesizing image information 
data synthesizedwith the input image information data contained 
in the input image signal, in synchronization with the input 
image signal in the switch circuit 22. For example, when the 

25 input image signal is an image signal corresponding to the NTSC 
standard, this input image signal is an image signal of an 
. interlace method, so that information for one screen is 
constituted by 2V (two periods) where one period of the vertical 
synchronizing signal is assumed to be IV. In such a case, based 

30 on the vertical synchronizing signal and the horizontal 
synchronizing signal, the address generation counter 12 
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generates the address information data in which two periods 
(2V) of the vertical synchronizing signal are considered to 
be one period. Further, when the input image signal is an image 
signal of a non-interlaced method, the address generation 
5 counter 12 generates the address information data in which one 
period (IV) of the vertical synchronizing signal is considered 
to be one period, according to the vertical synchronizing signal 
and the horizontal synchronizing signal. 
[0065] 

10 In addition, the address information data generation 

counter 12 may only generate the address information data as 
described above so as to be provided to the control unit 13, 
the first plane memory 18, the second plane memory 19, and the 
superposing order control unit 20, so that it may have any 

15 construction other than the construction as described above. 
[0066] 

The control unit 13 includes therein a control processing 
unit, an operation unit, a memory unit, etc. (which are not 
shown) , acquires a program and a data which are supplied from 

20 the outside of the image synthesizing apparatus 1 through an 
input terminal 14, controls a ROM 15 so as to acquire the program 
and the data which are stored in the ROM 15, or controls a drive 
16 so as to acquire the program and the data which .are read 
from a removable media 17 mounted at the drive 16. Then, based 

25 on the thus acquired program and data, the control unit 13 
controls the first plane memory 18, the second plane memory 
19, the superposing order control unit 20, and the switch 
circuits 21 and 22. 
[0067] 

30 In particular, based on the address information data 

supplied from the address generation counter 12, the control 
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unit 13 supplies the superposing order control unit 20 with 
the superposition control information data acquired as the data 
as described above or the superposition control information 
data generatedby implementing the programacquired as described 
5 above, to thereby control operation of the superposing order 
control unit 20 and operation of the switch circuits 21 and 
22 through the superposing order control unit 20. 
[0068] 

Further, based on the address information data supplied 
10 from the address generation counter 12, the control unit 13 
supplies the first plane memory 18 or the second plane memory 
19 with the synthesizing image information data which is 
acquired as the data as described above, non-compressed, and 
synthesized with the input image information data contained 
15 in the input image signal or the synthesizing image information 
data which is generated by implementing the program acquired 
as described above, to thereby control operation of the plane 
memories . 
[0069] 

20 In this case, it is preferable that the amount of data 

of the synthesizing image information data held by the first 
plane memory 18 at a time, the amount of data of the synthesizing 
image information data held by the second plane memory 19 at . 
a time, and the amount of data of the synthesizing image 

25 information data corresponding to the control of the 
superposition control information data (the synthesizing image 
information data being held by the superposing order control 
unit 20 at a time) are standardized to the amount of data for 
one screen of the same image size, for example, so that every 

30 data has the same amount, thus allowing the control unit 13 
to easily perform a control process and to reduce its load of 
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processing. 
[0070] 

The input terminal 14 may be connected with any apparatus, 
for example, an input apparatus, such as a keyboard, an imaging 
5 apparatus, such as a camera, a recording-medium playback 
apparatus, such as a disc player and a video-cassette recorder, 
or an image editing apparatus, such as a personal computer, 
etc. A variety of information data, such as an image information 
data inputted from such an apparatuses, and a control 
10 information data are supplied through the input terminal 14 
to the control unit 13. 
[0071] 

The ROM 15 prestores therein a program to be implemented 
in the control unit 13, data, such as the synthesizing image 
15 information data and the superposition control information data, 
etc. so as to be supplied to the control unit 13 according to 
the control by means of the control unit 13. 
[0072] 

In the drive 16, a removable media 17 constituted by a 
20 recording medium, such as a magnetic disk, an optical disc, 
a magneto-optical disc, a semiconductor memory, etc. ismounted 
as needed. Being controlled by the control unit 13, the drive 
16 reads the program or the data from the mounted removable 
media 17 so as to supply it to the control unit 13. 
25 [0073] 

The first plane memory 18 has, for example, a storage 
area constituted by a semiconductor memory, such as a DRAM 
(Dynamic Random Access Memory) and a control unit which controls 
input and output of the storage area, so that it holds the 
30 non-compressed synthesizing image information data which is 
supplied from the control unit 13 so as to be selectively 
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synthesized with the input image information data contained 
in the input image signal, for example, by one screen and outputs 
the thus held synthesizing image information data to the switch 
circuit 21, as requested by the switch circuit 21, according 
5 to the address information data supplied from the address 
generation counter 12 . 
[0074] 

For example, as shown in Fig. 2A, the first plane memory 
18 holds, the (bitmap form) non-compressed synthesizing image 

10 information data which is data on a pixel-by-pixel basis in 
the storage area 30 by one screen. At this stage, the first 
plane memory 18 respectively assigns 8 bits to the data 31 on 
a pixel-by-pixel basis of the synthesizing image information 
data so as to hold the synthesizing image information data where 

15 color information data of each pixel is expressed by 8 bits 
or less. Assuming that a number of effective pixels of the 
input image information data contained in the input image signal 
is 480 x 720 pixels, the first plane memory 18 uses an area 
of addresses (of 1-byte unit) OH through 54600H in the storage 

20 area 30, for example, so as to hold the synthesizing image 
information data of the same image size (480 x 720 pixels) as 
the image information data contained in the input image signal . 
[0075] 

When the switch circuit 21 requires to output the 
25 synthesizing image information data, the first plane memory 
18 reads a part, corresponding to the address information data, 
of the thus held synthesizing image information data and 
supplies it to the switch circuit 21. 
[0076] 

30 In addition, the amount of data of the synthesizing image 

information data which is held by the first plane memory 18 
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at a time may be larger or smaller than that of one screen, 
and may have any value. Further, a number of bits which is 
assigned by the first plane memory 18 to the data of each pixel 
of the synthesizing image information data may be 9 bits or 
5 more or may naturally be 7 bits or less. 
[0077] 

As shown in Fig. 2A, although it has been described that 
the first plane memory 18 holds the synthesizing image 
information data as data on a pixel-by-pixel basis in the 

10 non-compressed form (in bit map format) , alternatively it may 
hold the information data where the synthesizing image 
information data are arranged in a table according to an 
arrangement of pixels with the same color information data as 
shown in Fig. 2B, for example. 

15 [0078] 

In Fig. 2B the first plane memory 18 holds the table 40 
representing the synthesizing image information data supplied 
from the control unit 13. The table 40 is a table of the 
information data on a pixel line where continuous pixels of 

20 the same color information data are arranged in a group, and 
constituted by a start address column 41 which shows a start 
address of the pixel line on one screen, an end address column 
42 which shows an end address of the pixels line on one screen, 
and a color column 43 which shows color information data of 

25 the pixel line. 
[0079] 

For example, in the table 40 a row 4 4 shows that "cccc" 
is a color information data about a pixel line having addresses 
"aaaa" through, "bbbb" on one screen of an image corresponding 
30 to the synthesizing image information data. In this case, 
according to a request from the switch circuit 21, the first 
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plane memory 18 reads the information data in the color column 
of the row corresponding to address information data in the 
table 40 so as to be supplied to the switch circuit 21 as the 
synthesizing image information data- In addition the first 
5 plane memory 18 may naturally hold not only the color information 
data but also the information data arranged in a table including 
luminosity information data. 
[0080] 

The second plane memory 19 has the constitution similar 

10 to that of the first plane memory 18. Since the constitution 
as described above with reference to Fig. 2A is applicable also 
to that of the second plane memory 19, the description of its 
constitution will not be repeated herein. In addition, the 
first plane memory 18 and the second plane memory 19 may only 

15 hold the synthesizing image information data, so that their 
storage areas may be constituted by not only a semiconductor 
memory, a magnetic disk, an optical disc, a magneto-optical 
disc etc., for example. Further, they may naturally be 
constituted by other recording media . However, since the first 

20 plane memory 18 and the second plane memory 19 must store or 
output the synthesizing image information data within a short 
period of time, the storage areas need to be storage areas which 
can output and input information data at a somewhat high speed,, 
so that it is desirable to constitute them by semiconductor 

25 memories. 
[0081] 

Further, storage capacities of the first plane memory 
18 and the second plane memory 19 may have any amount . However, 
for example, if the storage capacities are so small as to store 
30 the synthesizing image information data only for one line, there 
is a possibility that a burden of a rewrite process by means 
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of the control unit 13 may become large. Therefore, it is 
generally desirable that the storage capacities of the first 
plane memory 18 and the second plane memory 19 are so large 
as to store the synthesizing image information data by one screen 
5 or more at a time. In other words, it is desirable that the 
storage areas of the first plane memory 18 and the second plane 
memory 19 are constituted by a DRAM which can provide a mass 
storage area more inexpensively than an SRAM (Static Random 
Access Memory) . 
10 [0082] 

The superposing order control unit 20 has the memory unit 
2 OA, stores the superposition control information data supplied 
to the control unit 13. Based on the superposition control 
information data and the address information data supplied from 

15 the address generation counter 12 , the superposing order control 
unit 20 controls the operation of the switch circuits 21 and 
22 and further controls a process of synthesizing the 
synthesizing image information data and the input image 
information data which are contained in the input image signal. 

20 [0083] 

For example, as shown in Fig. 3A the superposing order 
control unit 20 holds the superposition control information 
data 51, for one screen, which controls synthesis of the input 
image of the input image signal and the synthesizing image on 

25 a pixel-by-pixel basis in a storage area 50 of the memory unit 
20A. The superposition control information data 51 on a 
pixel-by-pixel basis is constituted by two bits of bits 61 and 
62, and is a control information data on a pixel-by-pixel basis 
for issuing instructions as to whether or not to synthesize 

30 the synthesizing image information data held in the first plane 
memory 18 or the second plane memory 19 with the input image 
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information data of the input image signal. Assuming that the 
number of effective pixels of the input image information data 
contained in the input image signal is 480 x 720 pixels, the 
superposing order control unit 20 holds the control information 
5 data corresponding to the same image size (480 x 720 pixels) 
as the input image information data contained in the input image 
signal, by means of the area of addresses (of 2-bit unit) OH 
through 54600H of the storage area 50, for example. 
[0084] 

10 In addition, it is desirable that the amount of data of 

the superposition control information data which is held by 
the superposing order control unit 20 is smaller than the amount 
of data of one synthesizing image information data, as shown 
in Fig. 3A, for example, because it is possible to reduce the 

15 load of processing of the control unit 13. In the case of Fig. 
3A, the amount of data for one pixel of the superposition control 
information data is 2 bits, and it is set so as to become smaller 
as compared with the amount of data of the synthesizing image 
information data where the data for one pixel consists of 8 

20 bits. 
[0085] 

Based on the address information data, the superposing 
order control unit 20 refers to the thus held superposition 
control information data on a pixel-by-pixel basis one by one, 

25 so as to control the operation of the switch circuits 21 and 
22. Therefore, the superposing order control unit 20 can 
control the image corresponding to the input image signal on 
a pixel-by-pixel basis. In addition it can selectively 
synthesize the image (part to which the position within a screen 

30 corresponds in the image of the synthesizing image information 
data) of the synthesizing image information data stored in 
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either the first plane memory 18 or the second plane memory 
19 in the arbitrary places in the screen of the image 
corresponding to the input image signal. 
[0086] 

5 In addition, the amount of data of the superposition 

control information data held by the superposing order control 
unit 20 at a time may be larger or smaller than that of one 
screen, and may have any value . Further, a number of bits which 
is assigned by the superposing order control unit 20 to each 
10 of the control information data 20A of a bit unit may be any 
bit. 
[0087] 

Based on the arrangement of pixels with the same image 
information data to be synthesized, such as may be shown in 

15 Fig. 3B, for example, other than the method as shown in Fig. 
3A, the superposing order control unit 20 may hold the 
information data (information data which are obtained by 
arranging, in a table, the transition points where the control 
changes) which are obtained by arranging the superposition 

20 control information data in a table. 
[0088] 

In Fig. 3B, the superposing order control unit 20 holds 
the table 70 representing the superposition control information 
data supplied from the control unit 13. The table 70 is a table 

25 of the superposition control information data for every pixel 
line which is obtained by arranging the control information 
data in groups, with respect to the continuous pixels of the 
same synthesizing image information data. The table is 
constituted by a start address column 71 which shows a start 

30 address of the pixel line on one screen, an end address column 
72 which shows an end address of the pixel line on one screen, 
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and a selected synthesizing image information data column 73 
which shows an synthesizing image information data selected 
in the pixel line. 
[0089] 

5 For example, in the table 70, a row 81 shows that the 

synthesis image information data which are synthesized with 
a pixel line having the addresses "aaaa" through "bbbb" on one 
screen of the image corresponding to the synthesizing image 
information data are the synthesizing image information data 

10 held in the first plane memory 18 . In this case, the superposing 
order control unit 20 refers to the information data on the 
selected synthesizing image information data column 73 of the 
row corresponding to the address information data in the table 
70, and controls the switch circuits 21 and 22 such that the 

15 synthesizing image information data may be synthesized with 
the input image information data contained in the input image 
signal . 
[0090] 

The switch circuit 21 is a switch circuit provided with 
20 two inputs and one output, so as to be supplied with the output 
from the first plane memory 18 or the second plane memory 19. 
Being controlled by the superposing order control unit 20, the 
switch circuit 21 switches connections on the input side so 
as to selectively connect either the first plane memory 18 or 
25 the second plane memory 19 to the switch circuit 22 , and requires 
the connected plane memory to output the synthesizing image 
information data. In other words, based on the control by means 
of the superposing order control unit 20, the switch circuit 
21 carries out the control such that the synthesis image 
30 information data may be supplied to the switch circuit 22 from 
either the first plane memory 18 or the second plane memory 
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19. 

[0091] 

The switch circuit 22 is a switch circuit provided with 
two inputs and one output, so as to be supplied with the image 
5 input signal and the output from the switch circuit 21. Being 
controlled by the superposing order control unit 20, the switch 
circuit 22 switches the connections on the input side, and 
selectively outputs either the supplied image input signal or 
the output through the switch circuit 21 (the output from the 
10 first plane memory 18 or the second plane memory 19) to the 
outside of the image synthesizing apparatus 1 as the output 
image signal. 
[0092] 

Now, the operation will be . described. The 
15 non-compressed input image signal inputted into the image 
synthesizing apparatus 1 is supplied to the synchronizing signal 
separation processing unit 11, and subjected to extraction of 
the synchronizing signals, such as the vertical synchronizing 
signal and the horizontal synchronizing signal. Based on the 
20 features, such as the amplitude, the frequency, etc., the 
synchronizing signal separation processing unit 11 extracts 
the synchronizing signals contained in the input image signal. 
The extracted synchronizing signal is supplied to the address 
generation counter 12 so as to be converted into the address 
25 information data which indicates a position on a screen and 
is supplied to the control unit 13, the first plane memory 18, 
the second plane memory 19, and the superposing order control 
unit 20. 
[0093] 

30 The control unit 13 acquires the program and the data 

through the input terminal 14 , or from the ROM 15 or the removable 
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media 17 mounted at the drive 16, and implements the program, 
for example, so that the synthesizing image information data 
is supplied to the first plane memory 18 or the second plane 
memory 19 at the timing based on the address information data 
5 supplied from the address generation counter 12. 
[0094] 

The control unit 13 supplies the first plane memory 18 
and the second plane memory 19 with the synthesizing image 
information data which are different from each other, whereby, 

10 as will be described hereafter, the images inserted into a part 
or the whole of the image corresponding to the input image signal 
can be switched without applying a load to the control unit 
13. In addition, the control unit 13 can supply a new 
synthesizing image information data to the plane memory which 

15 is not performing an output process of the synthesizing image 
information data- Therefore, the control unit 13 can supply 
the new synthesizing image information data at a timing other 
than a vertical blanking period of the input image information 
data contained in the input image signal, for example, a period 

20 when the other plane memory is performing the output process 
of the synthesizing image information data, whereby the load 
of the process of providing the synthesizing image information 
data can be reduced. Accordingly, performance necessary for 
the control unit 13 can be lowered, so that the manufacturing 

25 costs of the control unit 13 can be reduced. 
[0095] 

In addition, although it has been described that a number 
of plane memories is two in Fig. 1, it is not limited to this, 
and the number may naturally be three or more. The first plane 
30 memory 18 or the second plane memory 19 holds the synthesis 
image information data thus supplied by the control unit 13. 
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According to a request from the switch circuit 21, it outputs 
a part, corresponding to the address information data, of the 
held synthesizing image information data to the switch circuit 
21. 
5 [0096] 

Further, the control unit 13 acquires the program and 
the data through the input terminal 14, or from the ROM 15 or 
the removable media 17 mounted at the drive 16, and implements 
the program, for example, so that the superposition control 
10 information data is supplied to the superposing order control 
unit 20 at a timing based on the address information data supplied 
from the address generation counter 12. 
[0097] 

The superposing order control unit 20 holds, in the 
15 built-inmemory unit 20A, the superposition control information 
data supplied from the control unit 13, refers to the held 
superposition control information data, and controls the 
operation of the switch circuits 21 and 22 according to the 
address information data supplied from the address generation 
20 counter 12. Based on the control by means of the superposing 
order control unit 20, the switch circuit 21 reads either of 
the synthesizing image information data which are stored in 
the first plane memory 18 and the second plane memory ,19 so 
as to be supplied to the switch circuit 22 . Based on the control 
25 by means of the superposing order control unit 20, the switch 
circuit 22 selects either the input image signal or the output 
from the switch circuit 21, and outputs it to the outside of 
the image synthesizing apparatus 1 as the output image signal. 
[0098] 

30 In this way, the image synthesizing apparatus 1 

selectively synthesizes the synthesizing image information 
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data with the input image information data contained in the 
input image signal, so as to output is as the output image signal . 
Further, the image synthesizing apparatus 1 can easily 
synthesize the plurality of image information data, with the 
5 circuit size and the manufacturing costs remaining reduced. 
[0099] 

Next, with reference to flow charts in Fig. 4 and Fig. 
5, an image synthesis process by means of the image synthesizing 
apparatus 1 as shown in Fig. 1 will be described, further 
10 referring to Figs. 6A to 6C and Figs. 7A to 7C, as needed. 
[0100] 

Firstly, in step SI the control unit 13 determines whether 
or not to generate a first synthesizing image information data. 
When it is determined to be generated, the control unit 13 moves 

15 the process to step S2 so as to generate the first synthesizing 
image information data. Then, the control unit 13 supplies 
the first plane memory 18 with the generated first synthesizing 
image information data. Being supplied with the first 
synthesizing image information data, the first plane memory 

20 18 holds the supplied first synthesizing image information data 
in step S3, and moves the process to step S4. 
[0101] 

For example, when the input image corresponding to the 
input image information data contained in the input image signal 

25 is the input image 91 as shown in Fig. 6A, the control unit 
13 generates the first synthesizing image information data 
corresponding to a first synthesizing image 101 (as shown in 
Fig. 6B) of the same image size as the input image 91, so as 
to be supplied to the first plane memory 18. The first 

30 synthesizing image 101 is constituted by a area 102 containing 
characters "XYZ", a area 103 containing character "123", and 
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the other area 104. In other words, "the first synthesizing 
image 101 is an image for inserting the characters "XYZ" and 
"123" into the input image 91. In step S3, the first plane 
memory 18 holds the first synthesizing image information data 
5 corresponding to the supplied first synthesizing image 101. 
[0102] 

Further, in step SI when it is determined not to generate 
the first synthesizing image information data, the control unit 
13 controls the processes in steps S2 and S3 so as to be skipped, 

10 and moves the process to step S4 . In step S4 the control unit 
13 determines whether or not to generate the second synthesizing 
image information data . When it is determined to be generated, 
the control unit 13 moves the process to step S5 and generates 
the second synthesizing image information data. The control 

15 unit 13 supplies the generated second synthesizing image 
information data to the second plane memory 19 . Being supplied 
with the second synthesizing image information data, the second 
plane memory 19 holds, in step S6, the supplied second 
synthesizing image information data, and moves the process to 

20 step S7. . 
[0103] 

For example, the control unit 13 generates the second 
synthesizing image information data corresponding to a second 
synthesizing image 111 (as shown in Fig. 6C) of the same image 

25 size as the input image 91, so as to be supplied to the second 
plane memory 19. The second synthesizing image 111 is 
constituted by a area 112 containing the characters "ABC", a 
area 113 containing the character of "456" and the other area 
114. In other words, the second synthesizing image 111 is an 

30 image for inserting the characters "ABC" and "456" in the input 
image 91. 



[0104] 

For the sake of brevity in the description, the position 
and size of the area 112 in the second synthesizing image 111 
as shown in Fig. 6C are caused to correspond to those of the 
5 area 102 in the first synthesizing image 101 as shown in Fig. 
6B. Similarly, the position and size of the area 113 in the 
second synthesizing image 111 are caused to correspond to those 
of the area 103 in the first synthesizing image 101. The 
positions, sizes, etc. of these areas may naturally be 
10 independent of one another. 
[0105] 

In step S6 the second plane memory 19 hold the second 
synthesizing image information data corresponding to the 
supplied second synthesizing image 111. Further, in step S4 
15 when it is determined not to generate the second synthesizing 
image information data, the control unit 13 controls the 
processes in steps S5 and S6 so as to be skipped, and moves 
the process to step S7. 
[0106] 

20 In step S7 the control unit 13 determines whether or not 

to generate the superposition control information data which 
are control information data for synthesizing the synthesizing 
image information data with the input image information data 
contained in the input image signal (inserting the image 

25 corresponding to the synthesizing image information data into 
the image corresponding to the input image signal) . When it 
is determined to generate the superposition control information 
data, the control unit 13 moves the process to step S8, and 
generates the superposition control information data so as to 

30 be supplied to the superposing order control unit 20. Being 
supplied with the superposition control information data, the 
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superposing order control unit 20 holds, in step S9, the 
superposition control information data, and moves the process 
to step Sll of Fig. 5. 
[0107] 

5 For example, the control unit 13 generates the 

superposition control information data corresponding to the 
image 121 (as shown in Fig. 7A) of the same image size as the 
input image 91 so as to supply it to the superposing order control 
unit 20. The image 121 corresponding to superposition control 

10 information data is constituted by: a area 122 which exists 
in the same position and has the same size as the area 102 of 
the first synthesizing image 101 as shown in Fig. 6B, and the 
area 112 of the second synthesizing image 111 as shown in Fig. 
6C; a area 123 which exists in the same position and has the 

15 same size as the area 103 of the first synthesizing image 101 
as shown in Fig. 6B, and the area 113 of the second synthesizing 
image 111 as shown in Fig. 6C; and the other area 124 (which 
exists in the same position and has the same size as the area 
104 of the first synthesizing image 101 as shown in Fig. 6B, 

20 and the area 114 of the second synthesizing image 111 as shown 
in Fig. 6C. ) 
[0108] 

In Fig. 7A, the superposition control information data . 
are such that the first synthesizing image information data 

25 held in the first plane memory 18 is outputted into the area 
122 of the image 121, the second synthesizing image information 
data held in the second plane memory 19 is outputted into the 
area 123, and the input image information data is outputted 
into the area 124. Based on such superposition control 

30 information data, the superposing order control unit 2 0 controls 
the switch circuits 21 and 22, as will be described later, so 



as to output either the first synthesizing image information 
data, the second synthesizing image information data, or the 
input image information data . 
[0109] 

5 Now, returning to Fig. 4, in step S7 when it is determined 

not to generate the superposition control information data, 
the control unit 13 controls the processes in steps S8 and S9 
so as to be skipped, and moves, the process to step Sll of Fig. 
5. In step Sll of Fig. 5 the synchronizing signal separation 

10 processing unit 11 separates the synchronizing signal from the 
supplied input image signal, and supplies the separated 
synchronizing signal to the address generation counter 12. In 
step S12, based on the supplied synchronizing signal, the 
address generation counter 12 generates the address information 

15 data where one screen is considered to have one period, so as 
to be supplied to the control unit 13, the first plane memory 
18, the second plane memory 19, and the superposing order control 
unit 20. 
[0110] 

20 Having acquired the superposition control information 

data from the control unit 13, the superposing order control 
unit 20 specifies a pixel of attention based on the address 
information data in step S13, and determines, at the pixel of 
attention, whether or not to synthesize the first synthesizing 

25 image information data with the image information data contained 
in the input image signal. When it is determined to synthesize 
the first synthesizing image information data, the superposing 
order control unit 20 controls the switch circuits 21 and 22 
so as to output the first synthesizing image information data 

30 as the output image signal in step S14. In other words, based 
on the control by means of the superposing order control unit 
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20, the switch circuit 21 connects the first plane memory 18 
to the switch circuit 22, acquires the first synthesizing image 
information data from the first plane memory 18, and supplies 
it to the switch circuit 22. Based on the control by means 
5 of the superposition control unit 20 as described above, the 
switch circuit 22 connects the output from the switch circuit 
21 to the output side, and outputs the first synthesizing image 
information data supplied through the switch circuit 21 to the 
outside of the image synthesizing apparatus 1 as the output 
10 image signal. Having outputted the first synthesizing image 
information data, the switch circuit 22 moves the process to 
step S18. 
[0111] 

In step S13 when it is determined not to synthesize the 

15 first synthesizing image information data at the pixel of 
attention, the superposing order control unit 20 moves the 
process to step S15, and determines, at the pixel of attention, 
whether or not to synthesize the second synthesizing image 
information data with the image information data contained in 

20 the input image signal. When it is determined to synthesize 
the second synthesizing image information data, the superposing 
order control unit 20 moves the process to step S16, and controls 
the switch circuits 21 and 22 so^ as to output the. second 
synthesizing image information data as the output image signal . 

25 In other words, based on the control by means of the superposing 
order control unit 20, the switch circuit 21 connects the second 
plane memory 19 to the switch circuit 22 , and acquires the second 
synthesizing image information data f romthe secondplane memory 
19 so as to be supplied to the switch circuit 22. Based on 

30 the control by means of the superposition control unit 20 as 
described above, the switch circuit 22 connects the output from 
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the switch circuit 21 to the output side, and outputs the second 
synthesizing image information data supplied through the switch 
circuit 21 to the outside of the image synthesizing apparatus 
1 as the output image signal. Having outputted the second 
5 synthesizing image information data, the switch circuit 22 moves 
the process to step S18. 
[0112] 

In step S15 when it is determined not to synthesize the 
second synthesizing image information data at the pixel of 

10 attention, the superposing order control unit 20 moves the 
process to step S17, controls the switch circuits 21 and 22 
so as to output the input image information data contained in 
the input image signal as the output image signal. In other 
words, based on the control by means of the superposition control 

15 unit 20 as described above, to the output side the switch circuit 
22 connects the input on the side into which the input image 
signal is inputted, and outputs the input image information 
data contained in the input image signal which is inputted from 
the outside of the image synthesizing apparatus 1 to the outside 

20 of the image synthesizing apparatus 1 as the output image signal . 
Having outputted the input image information data, the switch 
circuit 22 moves the process to step S18. 
_ [0113] 

For example, according to the control by means of the 
25 superposing order control unit 20 based on the superposition 
control information data corresponding to the screen 121 as 
shown in Fig. 7A, the switch circuit 21 selectively outputs 
the first synthesizing image information data corresponding 
to the first synthesizing image 101 as shown in Fig. 6B and 
30 the second synthesizing image information data corresponding 
to the second synthesizing image 111 as shown in Fig. 6C, so 
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that the synthesizing image information data corresponding to 
the synthesizing image 131 as shown in Fig. 7B is outputted 
to the switch circuit 22. 
[0114] 

5 In Fig. 7B, the synthesizing image 131 is constituted 

by a area 132 corresponding to the area 122, a area 133 
corresponding to the area 123 , and a area 134 corresponding 
to the area 124. In the area 132, the image of the area 102 
of the first synthesizing image 101 is outputted. In the area 
10 133, the image of the area 113 of the second synthesizing image 
111 is outputted. In the area 134, neither the first 
synthesizing image 101 nor the second synthesizing image 111 
is outputted. 
[0115] 

15 According to the control by means of the superposing order 

control unit 20 based on the superposition control information 
data corresponding to the screen 121 as shown in Fig. 7A, the 
switch circuit 22 selectively outputs the input image 
information data corresponding to the input image 91 as shown 

20 in Fig. 6A and the synthesizing image information data 
corresponding to the synthesizing image 131 as shown in Fig. 
7B, so that the output image information data corresponding 
to the output image 141 as shown in Fig. 7C is outputted to 
the outside of the synthesis image apparatus 1 as the output 

25 image signal. 
[0116] 

In Fig. 7C the output image 141 is constituted by a area 
142 corresponding to the area 122, a area 143 corresponding 
to the area 123, and a area 144 corresponding to a area 124. 
30 In the area 142, the image of the area 102 of the first 
synthesizing image 101 is outputted. In the area 133, the image 
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of the area 113 of the second synthesizing image 111 is outputted. 
In the area 134, the image of the input image 91 is outputted. 
[0117] 

In this way, each part of the image synthesizing apparatus 
1 generates the output image information data which is 
selectively synthesized, for arbitrary areas, from the first 
synthesizing image information data, the second synthesizing 
image information data, and the input image information data, 
so as to be. outputted to the outside of the image synthesizing 
apparatus 1 as the output image signal. 
[0118] 

Now returning to Fig. 5, in step S18 the superposing order 
control unit 20 determines whether or not all the pixels in 
a frame, i.e. the data for one screen have been processed . When 
it is determined that the pixels for one period of address 
information data have not been processed and all the pixels 
in the frame have not been processed yet, it moves the process 
to step S19, pays attention to the next pixel according to the 
address information data, and returns the process to step S13, 
and the subsequent processes are repeated. In other words, 
each part of the image synthesizing apparatus 1 repeats the 
processes in step S13 through step S19 so that one frame of 
the image information data are processed on a pixel-by-pixel 
basis. In step S18 when it is determined that all the pixels 
in the frame have been processed, the superposing order control 
unit 20 moves the process to step S20. 
[0119] 

In step S20 the control unit 13 determines whether or 
not to terminate the image synthesis process. When the image 
information data corresponding to the following frame is 
inputted as the input image signal and it is determined not 
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to terminate the image synthesis process, the process is 
returned to step SI of Fig. 4, the subsequent processes are 
repeated. In other words, each part of the image synthesizing 
apparatus 1 repeats the processes in step SI through step S20 
5 so that the image synthesis process is performed for every frame 
with respect to all the input image information data contained 
in the input image signal. Thus, the image synthesizing 
apparatus 1 may, for every frame, generate the output image 
information data which are obtainedby selectively synthesizing 

10 the first synthesizing image information data, the second 
synthesizing image information data, and the input image 
information data, for every arbitrary area, so that the output 
image information data may be outputted to the outside of the 
image synthesizing apparatus 1 as the output image signal. In 

15 other words, for every arbitrary area, the image synthesizing 
apparatus 1 can switch the synthesizing images which are 
inserted in the input image . In this case, unlike a conventional 
apparatus, the control unit 13 does not create the synthesizing 
image information data or perform the control process so that 

20 the created synthesizing image information data may be 
synthesized with the input image information data . It may only 
supply the superposition control information data to the 
superposing order control unit 20 so as to easily switch the 
synthesizing images . 

25 [0120] 

In step S20, for example, when the input of the input 
image signal is stopped and it is determined to terminate the 
image synthesis process, the control unit 13 moves the process 
to step S21, and carries out a termination process so as to 
30 terminate the image synthesis process. In this way, each part 
of the image synthesizing apparatus 1 performs the image 
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synthesis process, whereby the image synthesizing apparatus 
1 can easily synthesize the plurality of image information data, 
with the circuit size and the manufacturing costs remaining 
reduced. 
[0121] 

Fig. 8 is a block diagram showing another example of 
structure of the image synthesizing apparatus to which the 
present invention is applied. In Fig. 8 the same reference 
numerals are used to indicate parts corresponding to Fig. 1 
and the description of them will not be repeated. 
[0122] 

In Fig. 8, the image synthesizing apparatus 150 is, for 
example, an apparatus in which a part or the whole of an image 
corresponding to the input image signal such as the image signal 
included in the television signal corresponding to the NTSC 
standard or the PAL standard is superimposed (or mixed) with 
another image showing a character, graphics, etc. , so that the 
resulting image is outputted as an output image signal. In 
other words, the image synthesizing apparatus 150 synthesizes 
and outputs the input image information data contained in the 
input image signal and the synthesizing image information data 
which is another image information data such that the image 
corresponding to the two image information data may be 
superimposed on (or mixed with) each other. At this stage, 
the image synthesizing apparatus 150 can synthesize the image 
information data such that, in the plurality of areas in the 
screen of the image corresponding to the input image signal, 
the character, graphics, etc. may be synthesized independently 
of each other for every area. For example, image synthesizing 
apparatus 150 can change a part of other images on which the 
image corresponding to the input image signal is superimposed 
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(or mixed) , further switch it to another image, or move it to 
another position. Also, the image synthesizing apparatus 150 
can change tints of the two superimposed (mixed) images. 
[0123] 

5 Similar to the control unit 13 of Fig. 1, the control 

unit 151 of the image synthesizing apparatus 150 includes the 
control processing unit, the operation unit, the memory unit, 
etc., which are not shown but built in. The control unit 151 
acquires the programand data which are supplied from the outside 

10 of the image synthesizing apparatus 150 through the input 
terminal 14, controls the ROM 15 so as to acquire the program 
and data which are stored in the ROM 15, or controls the drive 
16 so as to acquire the program and data which are read out 
of the removable media 17 mounted at the drive 16. Based on 

15 the thus acquired program and data, the control unit 150 controls 
the first plane memory 18, the second plane memory 19, the 
synthesizing ratio control unit 152, and the multipliers 153 
through 155 . 
[0124] 

20 In particular, based on the address information data 

supplied from the address generation counter 12, the control 
unit 151 supplies the synthesizing ratio control unit 152 with 
the synthesizing ratio control information data acquired as 
the data as described above or the synthesizing ratio control 

25 information data generated by performing the program acquired 
as described above, so as to control operation of the 
synthesizing ratio control unit 152 and operation of the 
multipliers 153 through 155 via the synthesizing ratio control 
unit 152. 

30 [0125] 

In addition, similar to the control unit 13 of Fig. 1, 
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the control unit 151 supplies the non-compressecl synthesizing 
image information data which is acquired as data in the manner 
as described above and is synthesized with the input image 
information data contained in the input image signal, or the 
5 synthesizing image information data generated by performing 
the thus acquired program, to the first plane memory 18 or the 
second plane memory 19, based on the address information data 
supplied from the address generation counter 12, so that 
operation of the first plane memory 18 and the second plane 
10 memory 19 is controlled. 
[0126] 

In this case, it is preferable that the amount of data 
of the synthesizing image information data held by the first 
plane memory 18 at a time, the amount of data of the synthesizing 

15 image information data held by the second plane memory 19 at 
a time, and the amount of data of the synthesizing image 
information data corresponding to control of the synthesizing 
ratio control information data held by the synthesizing ratio 
control unit 152 at a time are standardized to the amount of 

20 data for one screen of the same image size, for example, so 
as to be the same amount of data-, thus allowing the control 
unit 151 to easily perform a control process and to reduce its 
load of processing . 
[0127] 

25 The synthesizing ratio control unit 152 has the memory 

unit 152A, and stores the synthesizing ratio control information 
data supplied from the control unit 151. Based on the 
synthesizing ratio control information data, the address 
information data supplied from the address generation counter 

30 12, and the synthesis image information data supplied from the 
first plane memory 18 or the second plane memory 19, the 
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synthesizing ratio control unit 152 controls operation of the 
multipliers 153 through 155, and controls synthesis of the input 
image information data contained in the input image signal and 
the synthesizing image information data. 
5 [0128] 

As shown in Fig. 9A, for example, the synthesizing ratio 
control unit 152 holds, in a storage area 170 of its memory 
unit 152A, one screen of the synthesizing ratio control 
information data 171 which controls synthesis (mixture) of the 

10 input image of the input image signal and the synthesizing image 
on a pixel-by-pixel basis. The synthesizing ratio control 
information data 171 on a pixel-by-pixel basis is constituted 
by 6 bits and is the control information data on a pixel-by-pixel 
basis for specifying synthesizing ratios of the synthesizing 

15 image information data held at the first plane memory 18 and 
the second plane memory 19 and those of the input image 
information data contained in the input image signal. In the 
case of Fig. 9A, a bit group 181, which is two bits out of 6 
bits constituting the synthesizing ratio control information 

20 data 171 on a pixel-by-pixel basis, holds the information data 
for specifying the synthesizing ratio of the synthesizing image 
information data held at the first plane memory 18; a bit group 
182 which is similarly constituted by 2 bits, holds the 
information data for specifying the synthesizing ratio of the 

25 synthesizing image information data held at the second plane 
memory 19; further, a bit group 183 which is constituted by 
the remaining 2 bits holds the information data for specifying 
the synthesizing ratio of the input image information data. 
[0129] 

30 in other words, in this case, the synthesizing ratios 

of the synthesizing image information data held at the first 
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plane memory 18 and the second plane memory 19 and those of 
the input image information data contained in the input image 
signal can respectively be set by way of four grades. In 
addition, the setup of the synthesizing ratio for each image 
5 information data may be carried out by means of any number of 
grades, so that any number of bits may be assigned to the 
specification of the synthesizing ratio for each image 
information data. Further, the number of bits assigned to the 
specification of the synthesizing ratio for each image 
10 information data may naturally be a mutually different number . 
In other words, in Fig. 9A, any number of the bits may constitute 
the synthesizing ratio control information data 171 on a 
pixel-by-pixel basis . 
[0130] 

15 In addition, the amount of data of the synthesizing ratio 

control information data itself held by the synthesizing ratio 
control unit 152 (the amount of data by way of the number of 
the bits which constitute the synthesizing ratio control 
information data 171 on a pixel-by-pixel basis) may naturally 

20 be any size, however, the smaller possible one can more 
effectively reduce the load of processing of the control unit 
151. As described above, the larger the amount of data of 
synthesizing ratio control information data (the amount of data 
by the number of the bits which constitute the synthesizing 

25 ratio control information data 171 on a pixel-by-pixel basis) 
is, the more finely the specification of the synthesizing ratio 
for each image information data can be set up. 
[0131] 

The synthesizing ratio control unit 152 refers to the 
30 thus held synthesizing ratio control information data on a 
pixel-by-pixel basis one by one based on the address information 
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data, controls the operation of the multipliers 153 through 
155 , and multiplies each image information data with a 
synthesizing ratio which is a coefficient according to each 
synthesizing ratio . Therefore, the synthesizing ratio control 
unit 152 can control the image corresponding to the input image 
signal on a pixel-by-pixel basis, and mix and synthesize the 
image of the synthesizing image information data stored by 
either the first plane memory 18 or the second plane memory 
19 (part, of the image of the synthesizing image information 
data, to which a position in a screen corresponds ) in an arbitrary 
position in the screen of the image corresponding to the input 
image signal. 
[0132] 

In addition, the amount of data of the synthesizing ratio 
control information data held by the synthesizing ratio control 
unit 152 at a time (the amount of data by way of the number 
of the synthesizing ratio control information data 171 on a 
pixel-by-pixel basis) may be larger or smaller than that of 
one screen, or may have any value. 
[0133] 

Further, other than the method as shown in Fig. 9A, the 
synthesizing ratio control unit 152 may hold the information 
data in table obtained by arranging the synthesizing ratio 
control information data according to the arrangement of the 
pixels with the same synthesizing ratio of the image information 
data to be synthesized, as shown in Fig. 9B, for example. In 
Fig. 9B the synthesizing ratio control unit 152 holds the table 
190 representing the synthesizing ratio control information 
data supplied from the control unit 151. The table 190 is a 
table of the synthesizing ratio control information data for 
every pixel line in which the control information data are 
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arranged in group with respect to the continuous pixels of the 
same synthesizing ratio of the image information data to be 
synthesized. It is constituted by a start address column 191 
which shows a start address of the pixel line on one screen, 
5 an end address column 192 which shows an end address of the 
pixel line on one screen, and a synthesizing ratio information 
data column 193 which shows a synthesizing ratio of the 
synthesizing image information data in the pixel line. 
[0134] 

10 A column which shows a synthesizing ratio for each image 

information data is provided in the synthesizing ratio 
information data column 193. In the case of Fig. 9B, the 
synthesizing ratio information data 193 is constituted by a 
first plane memory column 193A which shows the synthesizing 

15 ratio of the synthesizing image information data held at the 
first plane memory 18, a second plane memory column 193B which 
shows the synthesizing ratio of the synthesizing image 
information data held at the second plane memory 19, and an 
input image information data column 193C which shows the 

20 synthesizing ratio of input image information data. 
[0135] 

For example, in the table 190, a line 201 shows that with 
respect to the pixel line having the addresses "aaaa" through 
"bbbb", on one screen, of the image corresponding to the input 

25 image information data, the synthesizing image information data 
held at the first plane memory 18 is mixed at a synthesizing 
ratio of 30%, the synthesizing image information data held at 
the second plane memory 19 is mixed at a synthesizing ratio 
of 30%, and the input image information data is mixed a 

30 synthesizing ratio of 40%. 
[0136] 
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In such a case, the synthesizing ratio control unit 152 
refers, in the table 190, to the information data on the 
synthesizing ratio information data column 193 on a line 
corresponding to an address information data, controls the 
5 multiplier 153 through 155 so as to synthesize each image 
information data at a specified synthesizing ratio and to 
multiply each image information data with the synthesizing ratio 
according to each synthesizing ratio. 
[0137] 

10 In addition, the synthesizing ratio control unit 152 

refers to the synthesis image information data supplied from 
the first plane memory 18 or the second plane memory 19. As 
for these synthesis image information data, when a synthesis 
image does not exist in the pixel in the position corresponding 

15 to the current address information data, the multipliers 153 
or 154 is controlled so that the image information data may 
not be mixed. In other words, in this case, the synthesizing 
ratio control unit 152 controls the multiplier 153 or 154 
corresponding to the image information data so that the image 

20 information data may be multiplied by a synthesizing ratio 
having a value of "0". In this way, only for a part in which 
a synthesis image actually exists, the image synthesizing 
apparatus 1 can mix its image information data with the input 
image information data, thus preventing the output image from 

25 degrading in the quality of images. 
[0138] 

The multiplier 153 multiplies the synthesizing image 
information data supplied from the first plane memory 18 with 
the synthesizing ratio acquired from the synthesizing ratio 
30 control unit 152, and the resulting product is supplied to the 
adder 156. The multiplier 154 multiplies the synthesizing 
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image information data supplied from the second plane memory 
19 with the synthesizing ratio acquired from the synthesizing 
ratio control unit 152, and the resulting product is supplied 
to the adder 156 . The multiplier 155 multiplies the input image 
5 information data with the synthesizing ratio acquired from the 
synthesizing ratio control unit 152, and the resulting product 
is supplied to the adder 156. 
[0139] 

The adder 156 adds together the respective image 
10 information data supplied from the multipliers 153 through 155, 
and the resulting sum as an output image signal is outputted 
to the outside of the image synthesizing apparatus 150. 
[0140] 

Now, the operation will be described. The 
15 non-compressed input image signal inputted into the image 
synthesizing apparatus 150 is supplied to the synchronizing 
signal separation processing unit 11, and subjected to 
extraction of the synchronizing signals, such as the vertical 
synchronizing signal and the horizontal synchronizing signal. 
20 Based on the features, such as the amplitude, the frequency, 
etc., the synchronizing signal separation processing unit 11 
extracts the synchronizing signals contained in the input image 
signal. The extracted synchronizing signals are supplied to 
the address generation counter 12 so as to be converted into 
25 the address information data which indicate positions on a 
screen and are supplied to the control unit 151, the first plane 
memory 18, the second plane memory 19, and the synthesizing 
ratio control unit 152. 
[0141] 

30 The control unit 151 acquires the program and the data 

through the input terminal 14 , or from the ROM 15 or the removable 
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media 17 mounted at the drive 16, and implements the program, 
for example, so that the synthesizing image information data 
is supplied to the first plane memory 18 or the second plane 
memory 19 at the timing based on the address information data 
5 supplied from the address generation counter 12. 
[0142] 

The control unit 151 supplies the first plane memory 18 
and the second plane memory 19 with the synthesizing image 
information data which are different from each other, whereby 
10 the image mixed with a part or the whole of the image corresponding 
to the input image signal can be switched without applying a 
load to the control unit 151. 
[0143] 

In addition, the control unit 151 can supply a new 

15 synthesizing image information data to the plane memory which 
is not performing the output process of the synthesizing image 
information data. Therefore, the control unit 151 can supply 
the new synthesizing image information data to the plane memory, 
for example, in which the output process of the held synthesizing 

20 image information data has been finished at a timing other than 
the vertical blanking period of the input image information 
data contained in the input image signal, so that the load of 
provision processing of the synthesizing image information data 
can be reduced. Accordingly, performance necessary for the 

25 control unit 13 can be lowered, so that manufacturing costs 
of the control unit 151 can be reduced. In addition, although 
it has been described that the number of plane memories is two 
in Fig. 8, it is not limited to this, and the number may naturally 
be three or more. 

30 [0144] 

Being supplied with the synthesis image information data 
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from the control unit 151, the first plane memory 18 and the 
second plane memory 19 hold the synthesis image information 
data, so that at a timing based on the supplied address 
information data, part of the held synthesizing image 
5 information data, the part corresponding to the address 
information data, are outputted to the multiplier 153 or 154 
and supplied to the synthesizing ratio control unit 152. 
[0145] 

Further, the control unit 151 acquires the program and 
10 the data through the input terminal 14, or from the ROM 15 or 
the removable media 17 mounted at the drive 16, and implements 
the program, for example, so that the synthesizing ratio control 
information data is supplied to the synthesizing ratio control 
unit 152 at a timing based on the address information data 
15 supplied from the address generation counter 12. 
[0146] 

Being supplied with the synthesizing ratio information 
data from the control unit 151, the synthesizing ratio control 
unit 152 holds the synthesizing ratio information data in the 

20 built-in memory unit 152A. Based on the address information 
data supplied from the address generation counter 12 , it refers 
to the held synthesizing ratio control information data, and 
supplies the synthesizing ratio to the multipliers 153 through 
155 so as to control their operation. 

25 [0147] 

In this case, the synthesizing ratio control unit 152' 
refers to the synthesizing image information data supplied from 
the first plane memory 18 or the second plane memory 19 and 
determines whether or not an image to be mixed exists in the 
30 pixel corresponding to the current address information data. 
When it is determined not to be present, the synthesizing ratio 
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control unit 152 supplies. the synthesizing ratio having the 
value of "0" to the multiplier 153 or 154 and controls it so 
as not to mix the synthesizing images. In other words, with 
respect to the synthesizing ratio control information data, 
5 even in a area which is specif ied to be mixed with the synthesizing 
image information data, the synthesizing ratio control unit 
152 controls the synthesizing image information data so as not 
to be mixed with the synthesizing image when the images to be 
actually synthesized (for example, the character, graphics, 
10 etc.) do not exist in the synthesizing image. 
[0148] 

Being supplied with the synthesizing image information 
data from the first plane memory 18, the multiplier 153 
multiplies the synthesizing image information data with the 

15 synthesizing ratio supplied from the synthesizing ratio control 
unit 152, and the resulting product is supplied to the adder 
156. Being supplied with the synthesizing image information 
data from the second plane memory 19, the multiplier 154 
multiplies the synthesizing image information data with the 

20 synthesizing ratio supplied from the synthesizing ratio control 
unit 152, and the resulting product is supplied to the adder 
156 . Being supplied with the input image information data (the 
input image signal), the multiplier 155 multiplies the input 
image information data with the synthesizing ratio supplied 

25 from the synthesizing ratio control unit 152, and the resulting 
product is supplied to the adder 156. 
[0149] 

The adder 156 adds together the image information data 
supplied from the multipliers 153 through 155 on a 
30 pixel-by-pixel basis, generates the output image information 
data so as to be output ted to the outside of the image synthesizing 
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apparatus 150 as the output image signal. 
[0150] 

In this way, the image synthesizing apparatus 150 
synthesizes and mixes the synthesizing image information data 
5 with the input image information data contained in the input 
image signal so as to be outputted as the output image signal. 
Further, the image synthesizing apparatus 150 can easily 
synthesize the plurality of image information data, with the 
circuit size and the manufacturing costs remaining reduced. 
10 [0151] 

Next, with reference to flow charts of Fig. 10 and Fig. 
11, image synthesis processing by means of the image 
synthesizing apparatus 150 in Fig. 8 will be described, further 
referring to Figs. 12A to 12C, Figs. 13A to 13C and Figs. 14A 
15 to 14B as needed. 
[0152] 

Firstly, in step S41 the control unit 151 determines 
whether or not to generate the first synthesizing image 
information data. When it is determined to generate it, the 

20 control unit 151 moves the process to step S42 so as to generate 
the first synthesizing image information data. Then, the 
control unit 151 supplies the thus generated first synthesizing 
image information data to the first plane memory 18. Being_ 
supplied with the first synthesizing image information data, 

25 the first plane memory 18 holds, in step S43, the supplied first 
synthesizing image information data, and moves the process to 
step S44. 
[0153] 

For example, when the input image corresponding to the 
30 input image information data contained in the input image signal 
is an input image 211 as shown in Fig. 12A, the control unit 
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151 generates the first synthesizing image information data 
corresponding to a first synthesizing image 221 of the same 
image size as the input image 211 as shown in Fig. 12B, and 
supplies it the first plane memory 18. The first synthesizing 
5 image 221 is constituted by a area 222 containing a circular 
pattern (solid black) , a area 223 containing a pattern (in 
concentric circles) of a solid black doughnut, and the other 
area 224. In other words, the first synthesizing image 221 
is an image for synthesizing (or mixing) the input image 211 

10 with the patterns (in concentric circles) of the solid black 
doughnut and the solid black circle. In step S43 the first 
plane memory 18 holds the first synthesizing image information 
data corresponding to the supplied first synthesizing image 
221. 

15 [0154] 

In step S41 when it is determined not to generate the 
first synthesizing image information data, the control unit 
151 controls the processes in steps S42 and S43 so as to be 
skipped, and moves the process to step S44. 
20 [0155] 

In step S44 the control unit 151 determines whether or 
not to generate the second synthesizing image information data. 
When it is determined to generate it, the control unit 151 moves 
the process to step S45 and generates the second synthesizing 

25 image information data . The control unit 151 supplies the thus 
generated second synthesizing image information data to the 
second plane memory 19. In step S46, being supplied with the 
second synthesizing image information data, the second plane 
memory 19 holds the supplied second synthesizing image 

30 information data, and moves the process to step S47. 
[0156] 
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For example, as shown in Fig. 12C, the control unit 151 
generates the second synthesizing image information data 
corresponding to the second synthesizing image 231 of the same 
image size as the input image 211, and supplies it to the second 
5 plane memory 19. The second synthesizing image 231 is 
constituted by a area 232 containing a solid black triangle, 
a area 233 containing a solid black quadrangle, and the other 
area 234. In other words, the second synthesizing image 231 
is the image for synthesizing (or mixing) the input image 231 
10 with the solid black triangle and the solid black quadrangle. 
[0157] 

For the sake of brevity in the description, the position 
and size of the area 232 in the second synthesizing image 231 
as shown in Fig. 12C are caused to correspond to those of the 

15 area 222 in the first synthesizing image 221 as shown in Fig. 
12B. Similarly, the position and size of the area 233 in the 
second synthesizing image 231 are caused to correspond to those 
of the area 223 in the first synthesizing image 221. The 
positions, sizes, etc. of these areas may naturally be 

20 independent of one another. 
[0158] 

In step S46 the second plane memory 19 holds the second 
synthesizing image information data corresponding to the 
supplied second synthesizing image 231. In step S44 when it 
25 is determined not to generate the second synthesizing image 
information data, the control unit 151 controls the processes 
in steps S45 and S4 6 so as to be skipped, and moves the process 
to step S47. 
[0159] 

30 In step S47 the control unit 151 determines whether or 

not to generate the synthesizing ratio control information data 
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which are the control information data for synthesizing the 
input image information data contained in the input image signal 
with the synthesizing image information data (for mixing the 
image corresponding to the input image signal with the image 
5 corresponding to the synthesizing image information data) . 
When it is determined to generate the synthesizing ratio control 
information data, the control unit 151 moves the process to 
step S48, and generates the synthesizing ratio control 
information data so as to be supplied to the synthesizing ratio 

10 control unit 152. In step S49, being supplied with the 
synthesizing ratio control information data, the synthesizing 
ratio control unit 152 holds the synthesizing ratio control 
information data, and moves the process to step S51 of Fig. 
11. 

15 [0160] 

For example, as shown in Fig. 13A, the control unit 151 
generates the synthesizing ratio control information data 
corresponding to the image 241 of the same image size as the 
input image 211 of a pattern hatched diagonally upward to the 

20 right in the figure, and supplies it to the synthesizing ratio 
control unit 152. The image 241 corresponding to synthesizing 
ratio control information data is divided into a area 242, a 
area 243, and a area 244 which is the other area. In each area, 
the synthesizing ratio of image information data is specified 

25 independently of one another. 
[0161] 

The position and size of the area 242 correspond to those 
of the area 222 of the first synthesizing image 221 as shown 
in Fig. 12B and those of the area 232 of the second synthesizing 
30 image 231 as shown in Fig. 12C. The position and size of the 
area 243 correspond to those of the area 223 of the first 
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synthesizing image 221 as shown in Fig. 12B and those of the 
area 233 of the second synthesizing image 231 as shown in Fig. 
12C. The position and size of the area 242 correspond to those 
of the area 224 of the first synthesizing image 221 as shown 
5 in Fig. 12B and those of the area 234 of the second synthesizing 
image 231 as shown in Fig. 12C. 
[0162] 

In the area 242, the synthesizing ratio control 
information data corresponding to the image 241 as shown in 

10 Fig. 13A is set up so as to mix and output 50% of the first 
synthesizing image information data, 30% of the second 
synthesizing image information data, and 20% of the input image 
information data. Further, in the area 123, this synthesizing 
ratio control information data is set up so as to mix and output 

15 50% of the second synthesizing image information data and 50% 
of the input image information data. Further, in the area 124 
this synthesizing ratio control information data is set up so 
as not to mix the synthesizing image information data but to 
output the input image information data only. 

20 [0163] 

Based on such synthesizing ratio control information data, 
the synthesizing ratio control unit 152 controls the multipliers 
153 through 155 by supplying the. synthesizing ratios, and 
outputs the image which is obtained by mixing the first 
25 synthesizing image information data, the second synthesizing 
image information data, and the input image information data 
at a predetermined ratio. 
[0164] 

Now, returning to Fig. 10, in step S47 when it is determined 
30 not togenerate the synthesizing ratio control information data, 
the control unit 152 controls the processes in steps S48 and 
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S49 so as to be skipped, and moves the process to step S51 of 
Fig. 11. In step S51 of Fig. 11, the synchronizing signal 
separation processing unit 11 separates the synchronizing 
signal from the supplied input- image signal, and the separated 
5 synchronizing signal is supplied to the address generation 
counter 12. In step S52, based on the supplied synchronizing 
signal, the address generation counter 12 generates the address 
information data in which one screen is considered to be one 
period, and supplies it to the control unit 151, the first plane 
10 memory 18, the second plane memory 19, and the synthesizing 
ratio control unit 152. 
[0165] 

In step S53, having acquired the synthesizing ratio 
control information data from the control unit 151, the 

15 synthesizing ratio control unit 152 specifies a pixel of 
attention based on the address information data. Based on the 
synthesizing ratio control information data, the synthesizing 
ratio control unit 152 determines, at the pixel of attention, 
whether or not to mix the first and second synthesizing image 

20 information data with the image information data contained in 
an input image signal. When it is determined to mix it with 
the first or second synthesizing image information data, the 
synthesizing ratio control unit 152 moves the process to step 
S54, and determines, at the pixel of attention, whether or not 

25 the synthesizing image (the character or graphics to be 
synthesized) exists, according to the first or second 
synthesizing image information data acquired from the first 
plane memory 18 or the second plane memory 19. 
[0166] 

30 In step S54 when it is determined that the synthesizing 

image exists, the synthesizing ratio control unit 152 moves 
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the process to step S55. Based on synthesizing ratio control 
information data, it supplies the synthesizing ratio to the 
multipliers 153 through 155, and controls the first or second 
synthesizing image information data so as to be synthesized 
5 with the input image information data. In other words, the 
synthesizing ratio control information data 152 supplies the 
multipliers 153 through 155 with- the synthesizing ratio of the 
value based on the synthesizing ratio control information data . 
The multiplier 153 multiplies the first synthesizing image 

10 information data with the supplied synthesizing ratio, and the 
resultingproduct is outputted to the adder 156 . The multiplier 
154 multiplies the second synthesizing image information data 
with the supplied synthesizing ratio, and the resulting product 
is outputted to the adder 156, Then, the multiplier 155 

15 multiplies the input image information data with the supplied 
synthesizing ratio, and the resulting product is outputted to 
the adder 156. Having controlled the multipliers 153 through 
155, the synthesizing ratio control unit 152 moves the process 
to step S57. 

20 [0167] 

In addition, in step S53 when it is determined, at the 
pixel of attention, not to mix the first or second synthesizing 
image information data with the image information data contained 
in the input image signal, the synthesizing ratio control unit 

25 152 moves the process to step S56 and supplies the synthesizing 
ratio to the multipliers 153 through 155, so as not to synthesize 
the synthesizing image information data but to output the input 
image information data only, whereby respective image 
information data are multiplied by the synthesizing ratio . In 

30 other words, the synthesizing ratio control information data 
152 supplies the multipliers 153 and 154 with the synthesizing 
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ratio having the value of "0", and supplies the multiplier 155 
with the synthesizing ratio having a value of "1." The 
multipliers 153 through 155 multiply the supplied synthesizing 
ratios with respective image information data, and outputs the 
5 resulting products to the adder 156. Therefore, in this case 
the adder 156 is supplied with a signal having the value of 
"0" from the multipliers 153 and 154 and supplied with the input 
image information data as it is from the multiplier 155. Upon 
completion of the process in step S56, the synthesizing ratio 
10 control unit 152 moves the process to step S57 . 
[0168] 

In step S54 when it is determined that the synthesizing 
image does not exist, the synthesizing ratio control unit 152 
moves the process to step S56. As described above, in order 

15 not to synthesize the synthesizing image information data but 
to output the input image information data only, the 
synthesizing ratio is supplied to the multipliers 153 through 
155 so that respective image information data are multiplied 
by the synthesizing ratios. Upon completion of the processing 

20 in step S56, the synthesizing ratio control unit 152 moves the 
process to step S57. 
[0169] 

In step S57, being supplied with the respective image 
information data after being multiplied by the synthesizing 
25 ratio, the adder 156 synthesizes the respective image 
information data so as to be outputted to the outside of the 
image synthesizing apparatus 150 as the output image signal. 
[0170] 

For example, the multiplier 153 multiplies first 
30 synthesizing image information data corresponding to the first 
synthesizing image 221 (as shown in Fig. 12B) with the 
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synthesizing ratio supplied from the synthesizing ratio control 
unit 152. As shown in the image 241 of Fig. 13A, in the first 
synthesizing image 221 the multiplier 153 multiplies the image 
information data of a pixel located in the area 222 with the 
5 synthesizing ratio having a value of "0.5", and multiplies the 
image information data of a pixel located in the area 223 and 
the area 224 with the synthesizing ratio having the value of 
" 0 . " 
[0171] 

10 In this way, the multiplier 153 generates the first 

synthesizing image information data corresponding to the first 
synthesizing image 251 as shown in Fig. 13B. The first 
synthesizing image 251 is constituted by a area 252 containing 
a circular pattern of horizontal lines, a area 253 containing 

15 a doughnut pattern (concentric circles) of horizontal lines, 
and the other area 254. 
[0172] 

The area 252 is a area corresponding to the area 222 of 
Fig. 12B. The solid black circle in the area 222 is multiplied 

20 by the synthesizing ratio so as to be a circular pattern of 
horizontal lines in the area 252. Therefore, the position and 
size of this circular pattern of horizontal lines are the same 
as those of the solid black circle in the area 222. The area 
253 is a area corresponding to the area 223 of Fig. 12B. The 

25 solid black doughnut-like pattern (concentric circles) in the 
area 222 is eliminated by multiplying the synthesizing ratio 
having the value of "0", so that the pattern does not exist 
in the area 253. The area 254 is a area corresponding to the 
area 224 of Fig. 12B. The multiplier 153 supplies the thus 

30 obtained first synthesizing image information data to the adder 
156. 
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[0173] 

For example, the multiplier 154 multiplies the second 
synthesizing image information data corresponding to the second 
synthesizing image 231 as shown in Fig. 12C with the synthesizing 
5 ratio supplied from the synthesizing ratio control unit 152. 
As shown in the image 241 of Fig. 13A, the multiplier 154 
multiplies the image information data of a pixel located in 
the area 232 in the second synthesizing image 231 with the 
synthesizing ratio having a value of "0 . 3" , multiplies the image 
10 information data of a pixel located in the area 223 with the 
synthesizing ratio having the value of "0.5", and multiplies 
the image information data of a pixel located in the area 224 
with the synthesizing ratio having the value of "0." 
[0174] 

15 In this way the multiplier 154 generates the second 

synthesizing image information data corresponding to the second 
synthesizing image 261 as shown in Fig. 13C. The second 
synthesizing image 261 is constituted by a area 262 containing 
a triangle of vertical lines, a area 263 containing a quadrangle 

20 of vertical lines, and the other area 264. 
[0175] 

The area 262 is a area corresponding to the area 232 of 
Fig. 12C. The solid black triangle in the area 232 is multiplied -~ 
by the synthesizing ratio so as to be the triangle of vertical 

25 lines in the area 262. Therefore, the position and size of 
this triangle of vertical lines are the same as those of the 
solid black triangle in the area 232. The area 263 is a area 
corresponding to the area 233 of Fig. 12C. The solid black 
quadrangle in the area 233 is multiplied by the synthesizing 

30 ratio so as to be the quadrangle of vertical lines in the area 
263. Therefore, the position and size of this quadrangle of 



vertical lines are the same as those of the solidblack quadrangle 
in the area 233. The area 264 is a area corresponding to the 
area 234 of Fig. 12C. The multiplier 154 supplies the thus 
obtained second synthesizing image information data to the adder 
5 156. 
[0176] 

Further, for example, the multiplier 155 multiplies the 
input image information data corresponding to the input image 
211 as shown in Fig. 12A with the synthesizing ratio supplied 

10 from the synthesizing ratio control unit 152. As shown in the 
image 241 of Fig. 13A, with respect to the input image 211, 
the multiplier 155 multiplies the image information data of 
a pixel located in the area 242 of the image 241 with the 
synthesizing ratio having a value of "0.2" , multiplies the image 

15 information data of a pixel located in the area 243 of the image 
241 with the synthesizing ratio of having the value of "0.5", 
and multiplies the image information data of a pixel located 
in the area 244 of the image 241 with the synthesizing ratio 
having the value of "1." 

20 [0177] 

In this way, the multiplier 155 generates the input image 
information data corresponding to the input image 271 as shown 
in Fig. 14A. The input image 271 is constituted by a area 272 
of a pattern hatched diagonally upward to the left in the figure, 
25 a area 273 of a pattern hatched diagonally upward to the left 
in the figure, and the other area 274 of a pattern hatched 
diagonally upward to the right in the figure. 
[0178] 

The area 272 is a area corresponding to the area 242 of 
30 Fig. 13A. In this area (272) the input image 211 the pattern 
hatched diagonally upward to the right in the figure is 

73 



multiplied by the synthesizing ratio so as to be a stripe pattern 
hatched diagonally upward to the left in the figure. The area 

273 is a area corresponding to the area 243 of Fig. 13A. In 
this area (273) the input image 211 of the pattern hatched 

5 diagonally upward to the right in the figure is multiplied by 
the synthesizing ratio so as to be a stripe pattern hatched 
diagonally upward to the left in the figure. Further, the area 

274 is a area corresponding to the area 244 of Fig. 13A. In 
this area (274), the input image 211 of the pattern hatched 

10 diagonally upward to the right in the figure has been multiplied 
by the synthesizing ratio having the value of "1", so that the 
image does not change. 
[0179] 

The multiplier 155 supplies the thus obtained input image 
15 information data to the adder 156. The adder 156 respectively 
acquires such image information data, adds and synthesizes them 
together so as to generate the output image information data 
corresponding to the output image 281 as shown in Fig. 14B. 
[0180] 

20 The output image 281 is an image obtained by adding and 

synthesizing the first synthesizing image 251 of Fig. 13B, the 
second synthesizing image 261 of Fig. 13C, and the input image 
271 of Fig. 14 A,, and _is_ constituted by the area 282 through 
area 284. The area 282 is a area corresponding to the area 

25 242 of Fig. 13A. The area 283 is a area corresponding to the 
area 243. The area 284 is a area corresponding to the area 
244 . 
[0181] 

Therefore, the area 282 includes the circular pattern 
30 of horizontal lines which exists in the area 252 of Fig. 13B, 
the triangle of vertical lines which exists in the area 262 



of Fig. 13C, and the pattern, of the area 272 in the input image 

271 of Fig. 14A, hatched diagonally upward to the left in the 
figure. In other words, the image of the area 282 is an image 
which is a mixture of the images of the area 252, the area 262, 

5 and the area 272 . At this stage, part where there are no patterns, 
such as a circle, a triangle, etc. in the area 252 and the area 
262 is not mixed with the images of the area 252 and the area 
262, so that the image of the area 282 is the image of the area 

272 as it is. 
10 [0182] 

The area 283 includes the quadrangle of vertical lines 
which exists in the area 263 of Fig. 13C and the pattern, of 
the area 274 in the input image 271 of Fig. 14A, hatcheddiagonally 
upward to the left in the figure. In other words, the image 

15 of the area 283 is an image which is a mixture of the images 
of the area 263 and the area 273. At this stage, part where 
the quadrangle of the area 263 does not exist is not mixed with 
the image of the area 263, so that the image of the area 283 
is the image of the area 273 as it is. 

20 [0183] 

In the area 284, the image of the pattern, of the area 
274 in the input image 271 of Fig . 14A, hatched diagonally upward 
to the right in the figure is outputted as it is . As described 
above, the adder 156 synthesizes the input image information 
25 data, the first and second synthesis image information data, 
so as to be outputted to the outside of the image synthesizing 
apparatus 150 as the output image signal. 
[0184] 

Now, returning to Fig. 11, in step S58 the synthesizing 
30 ratio control unit 152 determines whether or not all the pixels 
within a frame or the data for one screen have been processed. 



When it is determined that the pixels for one period of address 
information data have not been processed yet and all the pixels 
in the frame have not been processed, it moves the process to 
step S59. According to the address information data, it pays 
5 attention to the following pixel, and returns the process to 
step S53, and the subsequent processes are repeated. 
[0185] 

In other words, each part of the image synthesizing 
apparatus 150 processes one frame of the image information data 
10 on a pixel-by-pixel basis by repeating the process of step S53 
through step S59. In step S58 when it is determined that all 
the pixels in the frame have been processed, the synthesizing 
ratio control unit 152 moves the process to step S60. 
[0186] 

15 In step S60 the control unit 151 determines whether or 

not to terminate the image synthesis process. When the image 
information data corresponding to the following frame is 
inputted as an input image signal and it is determined not to 
terminate the image synthesis process, the process is returned 

20 to step S41 of Fig. 10, and the subsequent processes are repeated . 
[0187] 

In other words, by repeating the processes of step S41 
through step S60, each part of the image synthesizing apparatus 
150 performs the image synthesis process for every frame with 

25 respect to all the input image information data contained in 
the input image signal . Thus, the image synthesizing apparatus 
150 generates, for every frame, the output image information 
data synthesized by mixing the first synthesizing image 
information data, the second synthesizing image information 

30 data, and the input image information data for every arbitrary 
area, so as to output it to the outside of the image synthesizing 
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apparatus 150 as the output image signal. In other words, the 
image synthesizing apparatus 150 can switch the synthesizing 
images mixed with the input image for every arbitrary area. 
In this case, unlike a conventional apparatus, the control unit 
5 151 does not create the synthesizing image information data, 
or performs the control process so that the created synthesizing 
image information data may be synthesized with the input image 
information data. It may only supply the synthesizing ratio 
control information data to the synthesizing ratio control unit 
10 152 so as to easily switch the synthesizing images. 
[0188] 

In step S60, for example, when the input of the input 
image signal is stopped and it is determined to terminate the 
image synthesis process, the control unit 151 moves the process 
15 to step S61, and carries out the termination process so as to 
terminate the image synthesis process. 
[0189] 

In this way, each part of the image synthesizing apparatus 
150 performs the image synthesis process, whereby the image 
20 synthesizing apparatus 150 can easily synthesize the plurality 
of image information data, with the circuit size and the 
manufacturing costs remaining reduced. 
[0190] 

As described above, the image synthesizing apparatus to 
25 which the present invention is applied can easily synthesize 
the plurality of image information data by way of more various 
and complicated method, with the circuit size and the 
manufacturing costs remaining reduced. 
[0191] 

30 Next, an example in which such an image synthesizing 

apparatus is used as part of an apparatus will be described. 
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Fig. 15 is a block diagram showing an example of structure of 
a camcorder (Registered Trademark) to which the present 
invention is applied. 
[0192] 

5 In Fig. 15, the camcorder 300 images a photographic 

subject , stores the acquired image information data (moving 
image or still image) in a recording medium, or displays an 
image corresponding to the image information data on a display, 
during which the camcorder 300 synthesizes the image information 

10 data obtained by imaging with other images as described above. 
A camera unit 310 of the camcorder 300 is controlled by a control 
unit 340 as will be described later, performs an imagingprocess, 
generates the image information data, and supplies the image 
information data to a DSP (Digital Signal Processor) 320. 

15 [0193] 

In the camera unit 310, light from the photographic 
subject (not shown) is incident to a CCD (Charge Coupled Device) 
312 through a lens unit 311 constituted by a lens, an aperture 
mechanism, etc., and the light is converted into electrical 
20 signals. 
[0194] 

The image signal outputted by the CCD 312 is supplied 
to a pre-process circuit 3,13-.. The pre-process circuit 313 is 
constituted by a CDS (Correlated Double Sampling circuit) 

25 circuit, an AGC (Automatic Gain Control circuit ) circuit, and 
an A/D (Analog/Digital) converter, etc., which are not shown. 
The pre-process circuit 313 removes a noise component from the 
inputted image signal in the CDS circuit, adjusts a gain of 
the image signal in the AGC circuit, then converts the image 

30 signal which is an analog signal into a digital signal in the 
A/D converter, and outputs the digital signal to the DSP 320 
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as image information data. 
[0195] 

The camera unit 310 is provided with a timing generation 
circuit 314 which is controlled by a CPU (Central Processing 
5 Unit) 341 of the control unit 340 and generates a timing signal. 
The timing generation circuit 314 is controlled by the CPU 341 
so as to supply the timing signal to a driver 315 which controls 
operation of the lens unit 311, a driver 316 which controls 
operation of the CCD 312, and the CCD 312, respectively. 
10 [0196] 

Based on the supplied timing signal, the driver 315 
controls the operation of the lens unit 311 so as to adjust 
its aperture, zoom lens, shutter, etc. Based on the supplied 
timing signal, the driver 316 supplies a control signal to the 
15 CCD 312. Based on the control signal supplied from the driver 
316 and the timing signal supplied from the timing generation 
circuit 314, the CCD 312 carries out processes such as capture 
of the image signal etc. 
[0197] 

20 The DSP 320 is supplied with the image information data 

from the pre-process circuit 313, and includes therein an 
adjustment processing unit 321 which performs a process with 
respect to adjustment of . the image information - data, a 
compression/decompression processing unit 322 which 

25 compresses or decompresses the image information data, an SDRAM 
controller 323 which controls input and output of an SDRAM 
(Synchronous Dynamic Random Access Memory) 331 holding the image 
information data, etc. The DSP 320 causes the SDRAM 331 to 
hold the acquired image information data, as needed, which is 

30 subjected to digital signal processing, and supplies the 
processed image information data to the control unit 34 0. 
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[0198] 

Based on the image information data, the adjustment 
processing unit 321 generates control signals, such as an AF 
(Auto Focus) control signal, an AE (Auto Exposure) control 
5 signal, an AWB (Auto White Balance) control signal, etc., so 
that the control signals are supplied to the control unit 340 
through a bus 332. Further, the compression/decompression 
unit 322 compresses or decompresses the image information data 
by way of a predetermined compression/decompression method, 
10 during which the compression/decompression unit 322 causes the 
SDRAM 331 controlled by the SDRAM controller 323 to temporarily 
hold the image information data. 
[0199] 

Being subjected to the digital signal processing in this 
15 way, the image information data is supplied through the bus 
332 to the control unit 340 . The control unit 340 is constituted 
by the CPU- 341, a ROM 342, a RAM (Random Access Memory) 343, 
a clockcircuit 345, etc., and controls each part of the camcorder 
300. The CPU 341 controls each part or performs various types 
20 of processes according to a program stored in the ROM 342, or 
a program loaded into the RAM343 f romthe outside of the camcorder 
300 through the removable media 355 mounted at the drive 354 
or an external I/F (Inter-Face) 357. -The RAM 343 stores therein 
data required by the CPU 341 when implementing various types 
25 of processes, as needed. The RAM 343 temporarily stores therein 
the image information data etc. processed by each part. 
[0200] 

Based on instructions information data from a user which 
are inputted through an input unit 356, the control information 
30 data supplied from the DSP 320, the information data acquired 
by performing various types of programs, or the like, the CPU 
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341 controls the timing generation circuit 314, and controls 
the operation of the lens unit 311 or the CCD 312. According 
to a request from each part, the clock circuit 345 provides 
the present date, the present day of the week, the present time 
5 as well as imaging date and time etc. 
[0201] 

The CPU 341, the ROM 342, the RAM 343, and the clock circuit 
345 are mutually connected through the bus 344. This bus 344 
is also connected with the bus 332 to which the DSP 320 is 
10 connected. Further the bus 34 4 is connected with the image 
synthesis processing unit 351 which synthesizes the plurality 
of image information data. 
[0202] 

Details of structure of the image synthesis processing 

15 unit 351 are basically similar to those of the image synthesizing 
apparatus 1 as shown in Fig. 1 or of the image synthesizing 
apparatus 150 as shown in Fig. 8, and operates similarly, so 
that the block diagram of Fig. 1 or Fig. 8 is applicable. In 
other words, the image synthesis processing unit 351 performs 

20 processes similar to those in the case of the image synthesizing 
apparatus 1 or the image synthesizing apparatus 150 as described 
above so that the plurality of image information data are 
synthesized. In this case, as will be described later, the 
image synthesis processing unit 351 is controlled by the CPU 

25 341 through thebus 344 so as to perform the process . Therefore, 
unlike in the case of Fig. land Fig. 8, as for the image synthesis 
processing unit 351, the input terminal 14 may only be connected 
with the bus 344, so that the ROM 15 and the drive 16 (removable 
media 17) may not necessarily be provided. Further, in the 

30 case of the image synthesis processing unit 351, the input image 
signal is adapted to be inputted through the bus 344 and the 
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output image signal is adapted to be outputted to the bus 344. 
[0203] 

The bus 344 is further connected with an LCD control unit 
352 which controls operation of an LCD (Liquid Crystal Display) 
5 353 and the information data outputted thereto and inputted 
therein; a drive 354 to be equipped with the removable media 
355 including a magnetic recording medium (a flexible disk, 
a hard disk, a magnetic tape, etc.)/ an optical disc, a 
magneto-optical disc, a semiconductor memory, etc.; the input 
10 unit 356 which is subjected to operation by a user; and the 
external I/F 357 to which other apparatuses are connected. 
[0204] 

The LCD control unit 352 includes therein a VRAM (Video 
Random Access Memory) which is not shown. For example, the 

15 LCD control unit 352 stores, in the built-in VRAM, the image 
information data acquired from the DSP 320, the image 
information data acquired from the CPU 341, the image 
information data held at the RAM 343, the image information 
data acquired from the image synthesis unit 351, the image 

20 information data stored in the removable media 355, the image 
information data acquired through the external I/F 357, or the 
like, so that the image corresponding to the image data stored 
in the VRAM is displayed on the LCD 353. 
[0205] 

25 The drive 354 reads the computer program stored in the 

removable media 355 mounted at the drive 354, and supplies it 
to the CPU 341 which is caused to install it in the RAM 343 
etc. Into the removable media 355 mounted at the drive 354, 
the drive 354 stores various types of information data, such 

30 as the image information data, supplied from respective parts 
of the camcorder 300, such as for example the DSP 320, the CPU 



341, the RAM 343,. the image synthesis processing unit 351, the 

input unit 356, the external I/F 357, etc. 

[0206] 

The input unit 356 is constituted by various types of 
5 buttons, such as a shutter button and a menu button, a dial, 
a knob, a touch panel, etc. (none is shown) . Being operated 
by the user, it receives various instructions from the user, 
and supplies the instruction information data to each part of 
the camcorder 300, such as for example the CPU 341. 
10 [0207] 

The external I/F 357 is constituted by a connector in 
the shape in accordance with a predetermined standard, a driver 
for communication based on the standard, etc. , and is connected 
to another apparatus in a predetermined method. Another 

15 apparatus connected to the external I/F 357 communicates with 
the camcorder 300 via the external I/F 357 so as to exchange 
the data or the program. In addition, the external I/F 357 
has a predetermined communications antenna through which it 
may be connected with another apparatus by way of wireless 

20 communications via the antenna. 
[0208] 

In the camcorder 300 as described above, the image 
synthesis processing unit 351 is controlled by. the CPU 341 so 
as to acquire the image information data held in the RAM 343, 

25 the image information data supplied from the DSP 320, the image 
information data connected to the removable media 355 which 
is mounted at the drive 354, or the image information data 
supplied from another apparatus through the external I/F 357 
via the bus 344, during which in the case of a compressed image 

30 information data, the image synthesis processing unit 351 
supplies the image information data to the 
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compression/decompression unit 322 of the DSP 320, causes the 
image information data to be decompressed, and then acquires 
it. 

[0209] 

5 Having acquired the image information data, the image 

synthesis processing unit 351 inserts or mixes the synthesizing 
image corresponding to the synthesizing image information data 
supplied from the CPU 341 into an image corresponding to the 
image information data, for example, so that two image 
10 information data are synthesized in such a manner as described 
above • 
[0210] 

Then the image synthesis processing unit 351 supplies 
the synthesized image information data, through the bus 34 4, 

15 to the CPU 341 or the DSP 320 which is caused to perform the 
image processing, causes the RAM 343 to hold it, supplies it 
to the LCD control unit 352 which is caused to display the image 
corresponding to image information data on the LCD 353, supplies 
it to the drive 354 and causes the removable media 355 mounted 

20 at the drive 354 to store it, or supplies it to another apparatus 
through the external I/F 357. 
[0211] 

By providing the image synthesis processing unit 351 as 
described above and causing it to implement the image synthesis 

25 processing, the camcorder 300 can easily synthesize the image 
information data obtained by imaging or the image information 
data acquired from the outside with another image information 
data by way of various and complicated methods, and use the 
synthesized image information data, with the circuit size and 

30 the manufacturing costs remaining reduced. 
[0212] 



In addition, for example, when a still image is recorded 
on a recording medium, the still image can be one frame of a 
moving image . When the still image is displayed on an LCD etc . , 
the still image can be considered as a moving image in which 
images of the same frame follow one after another. Therefore, 
the present invention may similarly be applied to either the 
image information data of the moving image or the image 
information data of the still image. 
[0213] 

Although in the above description the present invention 
has been exemplified by the camcorder to which the present 
invention is applied, the present invention can be applied to 
other apparatuses, for example, image processing. apparatuses, 
such as a video camera, a digital still camera, a videocassette 
recorder, a television receiving and indicating apparatus, 
information processing apparatuses, such as a personal computer, 
a PDA (Personal Digital Assistants), etc., and communication 
apparatuses, such as a portable telephone, and the like. In 
addition, the present invention may be more effectively applied 
to a small apparatus, such as a mobile computing device, which 
has limits on its circuit size and manufacturing costs. 
[0214] 

In the above description, the address generation counter 
12 has been described as generating the address information 
data based on the synchronizing signal which is separated from 
the input image signal by the synchronizing signal separation 
processing unit 11. However, the present invention is not 
limited thereto, and the address information data may be 
generated based on any type of data. For example, the address 
generation counter 12 may include therein a clock signal 
generating circuit, so that the address information data may 
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be generated based on a clock signal generated by the clock 
signal generating circuit. Further, the address generation 
counter 12 may generate the address information data based on 
a clock signal supplied from the outside of image synthesis 
5 process apparatus. 
[0215] 

A series of processes as described above can be performed 
by means of hardware or by way of software. When they are 
performed by way of software, the present invention may be 

10 embodied by a personal computer as shown in Fig. 16, for example. 
In Fig. 16, a CPU 401 of a personal computer 400 performs various 
types of processes according to a program stored in a ROM 402 
or a program loaded from a memory unit 413 into a RAM 403. The 
RAM 403 suitably stores therein the data necessary for the CPU 

15 401 to perform various types of processes. 
[0216] 

TheCPU401, theROM402, and RAM 403 are mutually connected 
through the bus 404, with which an input/output interface 410 
is also connected. The input/output interface 410 is connected 

20 with an input unit 411 which may be a keyboard, a mouse, etc. , 
an output unit 412 which may be a speaker etc. as well as a 
display including a CRT (Cathode Ray tube), an LCD, etc., a 
_ memory unit 413 whichmay.be a hard disk etc . , and a communication 
unit 414 which may be a modem etc. The communication unit 414 

25 performs a communications process through a network including 
the Internet. 
[0217] 

The input/output interface 410 is also connected with 
a drive 415 where a removable media 421 is suitably mounted 
30 as needed which may be a magnetic disk, an optical disc, a 
magneto-optical disc, a semiconductor memory, or the like. A 
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computer program read out thereof is installed in the memory 
unit 413 as needed. When a series of processes are performed 
by way of software, the program which constitutes the software 
is installed over the network or from the recording medium. 
5 [0218] 

As shown in Fig. 16, the recording medium may not only 
be the removable media 421 having recorded therein a program 
which includes a magnetic disk (including a floppy disk), an 
optical disc (including a CD-ROM (Compact Disk-Read OnlyMemory) 

10 and a DVD (Digital Versatile Disk)), a magneto-optical disc 
(including an MD (Mini-Disk), a semiconductor memory, etc., 
and is delivered to a user for providing the program 
independently of the main part of the apparatus, but also be 
the ROM 402 or a hard disk contained in the memory unit 413, 

15. on which the program is recorded and which is assembled into 
the main apparatus beforehand so as to be provided for the user. 
[0219] 

In the specification, the steps which describes the 
program to be recorded on the recording medium may include not 
20 only the processes serially performed in accordance with the 
described order but also the processes performed in parallel 
or individually, so that the steps may not necessarily be 
processed serially.. _ 
[0220] 

25 Further in the specification, a system means the whole 

apparatus comprising a plurality of apparatus. 
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